engg-problems-cal-2

7/29/2019 engg-problems-cal-2

1/12

1

Engineering Computation ECL2-1

ENGINEERING COMPUTATION Lecture2

Stephen RobertsMichaelmas Term

Conditioning of simultaneous equations

Topics covered in this lecture:

1. Error types (revision)2. Ill-conditioning and the importance of quantifying it.3. Error metrics vector and matrix norms.4. The condition number.


Errors and Rounding (revision from first year)

Sadly, in most engineering problems quantities are not known accurately, butare subject to errors. These may be due to

Measurement inaccuracies. e.g. How accurately can you measure with

a rule? 0.5 mm? 0.1 mm?

Rounding errors are similar, but numbers are roundedto the nearestsignificant digit.

e.g., to 3 significant digits, both 2.504999 and 2.495001 round to 2.50,but is their difference really 0?

Programming Errors (mistakes) could be obvious, but can lead to subtle,small errors which propagate through a complex calculation.


2/12

2


Truncation errors. Whether using decimal or binary numbers, calculators,

computers, A/Ds etc. truncate numbers to a fixed number of digits.

e.g. A typical 12-bit A/D has 4096 different levels with an accuracy of

1 Least Significant Bit (LSB). On full scale this is a maximum error of1/4096 or 0.02%. which looks good.

But if an A/D with a full-scale range of 0 - 5.0 Volts were to be used to find

the difference between two voltages only 10 mV apart, which corresponds toabout 8 LSB, the maximum error due to the truncation of the reading by the

A/D would be

~ 2(voltage of 1 LSB) = 2(5.04096) = 2.44 mV

corresponding to 25% !

Most calculators work to 8 or 10 significant digits

Matlab floating-point numbers have a finite precision of roughly 16 significant

decimal digits

This may seem adequate, but can still give problems with ill-conditionedcalculations.


Quantifying errors for specific numerical problems.

The errors considered so far relate to general statements aboutmeasurement error and computer arithmetic accuracy.

Can we say more about errors associated with specific numericalproblems?

Yes!

Lets take a look at this, using the solution of linear simultaneousequations to illustrate how we can do this


3/12

3


Ill-conditioning


Ill-Conditioning

What are the effects of, say, rounding errors in the accuracy of solutions of thelinear simultaneous equation, bAx = ?

With simple one variable equations there are few problems.

e.g. consider the sensitivity of the solution of ax= bto errors in aand b.

Differentiating gives baxxa =+ ,

ora

a

b

b

a

a

b

b

x

x += .

So the proportional error in xis just the sum of the proportional errors in a and b.


4/12

4


Example of Ill-conditioning. consider matrix equation of type bAx = :

=

2

2

1.19.0

11

y

x

Assume a small error inthe second term of b.

When = 0, solution is x=1, y= 1.

If is included, the solutionis x= 1 + 5 , y= 1 - 5 .The proportional error hasbeen magnified by a factor

of 10 from /2 to 5 /1 !

Look at this graphically with

=0.1The two lines are almost

parallel, and a smallmovement of one due to moves the intersection ofthe line a long way!

How do we predict this ill-conditioning?

Ill-Con ditioned Equations

0

0.5

1

1.5

2

0 0.5 1 1.5 2

x

y

Solution moves

with error


Example of Ill-conditioning in an electrical circuit.

Consider a potential divider with aleakage resistor R2 (R2 >> R1). thevoltages and currents are connected bythe matrix equation

=

+

+

2

1

2

1

212

221

V

V

I

I

RRR

RRR

Apply equal voltages, but with an error : ( )+= 11 VV and ( )= 12 VV .

Solving gives ( )++

= 12 21

1RR

VI and ( )

+= 1

2 212

RR

VI where

1

21 2

R

RR += .

Error magnification factor gets larger as R2 gets larger, and the leakagedecreases!

R1

V1 V2

R1

R2

I1 I2


5/12

5


Example of Ill-conditioning in a structural problem.

This beam looks unsatisfactory!All is well if F1 = F2 =F : the beam is balanced, and N1 = N2 = F.

But if the applied forces F1, F2 are unbalanced, large reactions can bedeveloped!

Taking moments about an end, ( ) ( )baNbNbaF ++=+ 212 2 . Moments about theother end give a second equation, and the matrix equation for the reactions is:

=

+

+

+ 2

1

2

1

2

1

F

F

N

N

bab

bba

ba

Consider the case of almost balanced loads: ( )+= 11 FF and ( )= 12 FF .

Solving gives ( )+= 11 FN and ( )= 12 FN wherea

ba 2+= .

The unbalance factor gets much larger as the distance a between thesupports decreases!

F1 N1 N2 F2

b a b

a


6/12

6


Norms of vectors and matrices


Vector Norms

Many numerical algorithms are iterative and you use the distance between two

successive approximations to determine when to stop iterating.This poses the questionHow do we measure the size of this difference which can normally be expressed as a vector?

We could express it as the Euclidean distance :

( )

=++=

=

n

i

ixxx

1

22

2

2

12Lx This is called an l2-norm.

We can generalise this to an lp-norm defined bypn p

ipx

1

1

= x where pis an integer.

Useful norms are:

The l1-norm =n

ix

11

x , the sum of the magnitudes of the elements of vector x.

The l2-norm

2

1

1

2

2

=

n

ixx , the Euclidean length of vector x.

l , the infinity-norm ( )ipn p

ip

xx maxlim

1

1

=

=

x , the largest element of vector x.


7/12

7


These norms satisfy basic requirements for norms (check them for yourself!):

0=x if and only if all elements of x are zero.

xx kk =

yxyx ++ (triangle inequality)

Example:

for vector [ ]T12,4,3 , l1 = 19, l2 = 13 and l =12 .

This is all very well, but how can we extend the idea to enable us to definea (scalar) distance between two matrices?


Matrix Norms

To determine whether the equations bAx = are ill conditioned,we must be able to express the errors in a matrix A in the form of a matrix noranalogous to the vector norm.

If A is an NN matrix and x a vector of length N, then Ax is also of length N,and there is a number cfor which xAx c .

If 0> xthen,0x and so cx

Ax

You can see that cis a measure of the size of A.

Define the Matrix Norm 0xanyforx

Ax

A=

max

It turns out that this depends on which vector norms we used for Ax

and x .


8/12

8


It turns out that this depends on which vector norms we used for Ax and x .

The l1-norm gives

=

i

ijj

Amax1

A (max column sum).

The l-norm gives

=

j

iji

AmaxA (max row sum).

The Frobenius norm2

1

2

Fro

=

i j

ijAA (root-of sum-of-squares of the elements).


It can be shown that all of these matrix norms are compatible with the corresponding vectornorms, that

is xAAx . How do you prove this?

Here is the skeleton of the proof for the l1-norm:

Suppose Axy =

Then ==i i j

jijixAy

1y .

Since vuvu || ++ and vuuv = ,

then i j

jijxA

1y .

Reverse the order of summation to give

=

j i

ijj

j i

jijAxxA

1y .

Now for any columnj 1max A= iij

ji

ij AA ,

so1111

xAAxy =

The other proofs are similarly elegant but tedious!


9/12

9


Examples of matrix norms

=

223

512

321

A 101=A and 8=

A .

Use Matlab function inv(A) to find A-1 and get 9677.01

1=

A and

8710.01 =

A .

Note that 11 AA .

Example 2: For our original ill- conditioned matrix

=

1.19.0

11A 2

1=A .1 and

2=

A , but

=

55.4

55.51

A , 101

1=

A and 5.101 =

A


The condition number of a matrix


10/12

10


Condition Number

Use the lp-norm to define the condition number ( )ppp

k 1

= AAA .

The condition number is a measure of how ill-conditioned the equationbAx = is. It clearly can only be defined for a non-singular matrix.

Kreyzig (look it up in 7th edition pp 996-998) proves that , if there is an error matrix

A in A, then the corresponding error x in the solution x is given by

( )p

p

pk

p

p

A

AA

x

x .

In other words, kp(A) is the error multiplication factor and if large, indicates that the problem isill-conditioned.

Similarly, if there is a right hand side error b in the vector b, then the corresponding errorx in the solution x is given by

( )p

p

pk

p

p

b

bA

x

x .


Lets try this with bAx = given by

=

2

2

1.19.0

11

y

x .

A condition number is given by ( ) 21101.21

1

11===

AAAk , which is high.

Then ( )

25.54

21

1

11

1

1==

b

bA

x

xk , showing the large multiplication in error.

By contrast, you can show that in the well-conditioned equation

=

2

2

11

11

y

x,

the condition number is only 2.


11/12

11


The Infamous Hilbert Matrix

When fitting polynomials to data in a later lecture, we will come across the Hilbert

Matrix having elements1

1

+=

jiHij .

e.g.

=

7/16/15/14/1

6/15/14/13/1

5/14/13/12/1

4/13/12/11

4H .


Use MATLAB to print out the condition numbers of the first 10 Hilbert Matrices:

function hilbert(n)% Calculates the inf. cond. number for the first n Hilbert matrices.% mlgo 10/1/99

for i = 1:nA=hilb(i);k=cond(A);fprintf('k2H(%d)) = %f\n', i, k);

end


12/12

12


Run it:

hilbert(10)k2H(1)) = 1.000000k2H(2)) = 19.281470k2H(3)) = 524.056778k2H(4)) = 15513.738739k2H(5)) = 476607.250243k2H(6)) = 14951058.641724k2H(7)) = 475367356.277700k2H(8)) = 15257575253.665688k2H(9)) = 493153214118.786620k2H(10)) = 16025336322027.105000

Hilbert Matrices rapidly become ill-conditionedIn this particular application this causes problems when fitting high orderpolynomials to data


Summary

In this lecture we have considered the following topics:

1. Error types (revision)2. Ill-conditioning and the importance of quantifying it.3. Error metrics vector and matrix norms.4. The definition of the condition number.

(3) and (4) are especially important and we will meet them again as weconsider iterative solutions to different classes of numerical algorithm.

In the next lecture we begin to do this as we consider iterative solutions of

simultaneous equations.

Documents

engg-problems-cal-2