GENG2140 2010 Condition Number

8/8/2019 GENG2140 2010 Condition Number

1/12

1

Ill-conditioned matrices

Norm of a vector

Norm of a matrix

Condition number via matrix norm

A.V. Dyskin, CRE, UWA GENG2140 Slide 49

Estimation of errors in solving systems of

linear algebraic equations

Points to note

There exist systems of linear equations that

Theoretically have unique solution but

Practically cannot be solved using a computer with thegiven accuracy

Method of identification of such systems-condition number

Determination of condition number


Norm of a matrix

Condition number

Meaning of condition number


2/12

2

Detection of internal structureTransmitter/receiver

lection.

(1) (1)

vp(1)=2.564 km/s, vs

(1) =2.190 km/s

vp(2)=3.552 km/s, vs

(2) =3.034 km/s

Transitionzone.

Noref

x2

p , s

vp(2), vs

(2)Arrival times of the signals

)2(

2

)1(

1

)2(

2

)1(

1 22,22

ss

s

pp

pv

x

v

xt

v

x

v

xt

Arrival times: tp=1.343 s, ts=1.572 s


572.1659.0913.0

343.1563.0780.0

21

21

xx

xx

vp - compressive wave velocity

vs - shear wave velocity

Equations

The system

(Forsythe, Malcolm and Moler, 1977)

572.1

343.1

659.0913.0

563.0780.0

2

1

x

x

The exact answer is x= 1, 1

A x b Ax= b



3/12

3

SolutionGaussian elimination with partial pivoting on a computer with =10, t=3:

001.0

72.1

10

01

001.0

573.1

10

0913.0

001.0

572.1

10

659.0913.0

000001.0

572.1

001.00

659.0913.0

343.1

572.1

563.0780.0

659.0913.0

572.1

343.1

659.0913.0

563.0780.0

2

1

2

1

2

1

2

1

2

1

2

1

x

x

x

x

x

x

x

x

x

x

x

x

Partial pivoting


The solution: x*=(1.72, -0.001). The errore=x*-x=(-0.71, 1.001)

is of the order of the true solution.

Analysis of the Example

Higher accuracy of computation can improve

e s ua on. or ns ance, e so u on

obtained on a computer with t=6 is: (1.00000,

1.00000) which is very accurate.The residual is very small:


Determinant is very small: det(A)=0.000913

0023.0

.bAxr


4/12

4

Counter-example

The exact solution is x=(1, 1).

035.0

04.0

03.0005.0

01.003.0 1

x

x

Gaussian elimination on a computer with =10, t=3:

1

1

10

01

1

03.0

10

003.0

1

04.0

10

01.003.0

0283.0

04.0

0283.00

01.003.0

035.0

04.0

03.0005.0

01.003.0

11

2

1

2

1

2

1

x

x

x

x

x

x

x

x

x

x


The solution: x*=(1, 1) is very accurate.

However, the determinant of the matrix is even less than in the Example.

Images of unit vectors for the

matrix from the Counter-exampleUnit vectors:

0.04

03.0005.0

01.003.0A

1

0

36

72108

144

180

216324

o

oo

o

o

o

o

o

0.02

-0.04 -0.02 0 0.02 0.04

0o

36o72

o

108o

144o

180o

max |Au|= 0.037min |Au|= 0.023

v=Auu=(cos , sin ), o o


252 288o o

-0.04

-0.02216o

252o

288o

324

max |Au|

min |Au|


5/12

5

Images of unit vectors for thematrix from the Example

Unit vectors:

563.0780.0

A

0

0.5

1

-0.8 -0.4 0 0.4 0.81

0

36

72108

144

180oo

o

o

o

o

o

0o

36o

72o

108o

288o

324o

u=(cos , sin ),

v=Au

max |Au|= 1.48

min |Au|= 1.6*10-6

o o..


-1

-0.5252 288

324o o

144o

180o

216o

252o

max |Au|

min |Au|

6

The condition number of a matrix

x

x

x

A

0max

xx

A1

max

x

x

x

A

0min

xx

A1

mincon

, ,

Matrices with high condition numbers are called ill-

conditioned, otherwise they are called well-conditioned.

(Here ||x|| denote a length of vector x.)


Properties

1. IfA is singular, 0min1

xx

A , hence, cond(A)=.

2. cond(A)1 3. cond(cA)=cond(A) for any scalar c


6/12

6

Questions

What does it mean "high condition number"?

Is there a better (easier) way to compute the

To answer these questions the notions ofnorm of vectorand

norm of matrix are necessary.

Norm of a vector, properties

Norm of a matrix based on the vector norm


Properties of matrix norm

The condition number via matrix norm

Meaning of the condition number

Norm of a vector

Norm of a vector x is a real number ||x|| such that:

yxyx

xx

0

xx

)(

scalarsallfor)(

0)(

0if0)(

iv

ccciii

ii

i

n

i

ix1

1

x


common

vector

norms:i

ni

n

i

i

x

x

,,1

1

2

2

max

x

x - Euclidian length


7/12

7

Examples

1 1.5131.01 x

1

3

1.0x

31,3,1.0,1max

32.31901.012

1

x

x


Although these three norms are different, the results are ofthe

same order of magnitude. The chose of a particular norm is

dictated by the convenience to perform the analysis.

Norm of a matrix

xAA max xAx 1 xx 0, ,

The particular value of the matrix norm is determined by a

particular chose of the vector norm. Thus, if

n

n

aaa

aaa

22221

11211

nn


nnnn aaa

21

t en

There is no simple formula for Euclidean norm.

j

ijni

i

ijnj

aa1

,,11

,,11max,max


8/12

8

Properties of matrix normFor any norm

BABAiv

cAccAiii

ii

AAi

)(

scalarsallfor)(

00)(

0if0)(

(v) For any matrixA and vector x ||Ax|| ||A|| ||x||


Moreover, there always exists a vector xA, such that ||AxA||= ||A|| ||xA||

(vi) For any two matricesA andB ||AB|| ||A|| ||B||

Example 1

1035.0

5.0121

5.95.325.95.3max

1115.0100135.132205.01max

1132

105.10

1

A

A


7

7

5.2

5.4

5.4

max

1132

105.10

1035.0

5.0121

max

A


9/12

9

Example 2

15.0121

11

5.0

5.2

5.1

1

1

1

105.10

1035.0

5.0121

41111,

1

1,5.9,

1132

105.10

.

x

xx

A

AA


111111,3845.9,5.735.05.25.1

311132

xxxx

AAAA

Example 3

1

0

1035.0

5.0121

11

3

2

1

0

1035.0

5.0121

1,

0

0,5.9,

1132

105.10

x

xx

A

AA


111111,5.915.9,5.935.132

3

.

01132

.

xxxx

AAAA


10/12

10

Condition number via matrixnorm

xA 1

Let y=A-1x, x=Ay. Then

y

yyx

y

yx AAA

0

00

1

min

maxmax

10

min

max

)(cond

AAA

A

Ay

y

y

y


Therefore1)(cond AAA

Condition number for the matrix

from Example and Counter-exampleThe Example

6

6

1

1

1

1

1066.2)(cond

1057.1,69.1,780000913000

563000659000,

659.0913.0

563.0780.0

A

AAAA

The Counter-example


884.1)(cond

1.47,04.0,3.3588.5

8.113.35,

03.0005.0

01.003.0

1

1

1

1

A

AAAA


11/12

11

Estimation of errors in solvinglinear algebraic equations 1

Ax=b

1. Error in the right partA(x+x)=b+b

x is the error in the solution, x. The error satisfies the systemAx=b

bxbxbxbx AAAA ,11

b

b

b

b

x

x

)(Cond1 AAA


b

b

x

x

)(Cond A

The condition number of a matrix is

the magnification factor of the

relative error.

Estimation of errors in solving

linear algebraic equations 2

Ax=b

2. Error in the matrix (A+A)x=bx is the error in the solution, x. The error satisfies the systemAx=b

AAA 1bbx

bbxbx 1 AAAA


AAAA ononon

bbx

The condition number of a matrix is the magnification

factor of the relative error.


12/12

12

Structure of this chapter

Ill-conditioned systems

and matrices

Condition number

Matrix normCondition number

via matrix normVector norm


Condition number as

error multiplicator

Summary Amongst systems of linear algebraic equations there exist

such that amplify errors in the matrix or right hand parts -ill-conditioned systems

-,slenderness of the image of a circle. Generally they arecharacterised by large condition number.

The condition number is defined as

x

xx

A

A

A1

min

max

)(cond


ropert es If detA=0, then cond(A)=; cond(A)1; cond(cA)=cond(A)

Method of computation through matrix norm

Meaning multiplication factor of errors in the right-handpart or the system.

Documents

GENG2140 2010 Condition Number