Numerical AnalysisCC413

Propagation of Errors

2

Propagation of Errors

In numerical methods, the calculations are not made with exact numbers. How do these inaccuracies propagate through the calculations?

Underflow and Overflow Numbers occurring in calculations that

have a magnitude less than 2 -1023 .(1+2 -52) result in underflow and are generally set to zero.

Numbers greater than 2 1024 .(2-2 -52) result in overflow.

Significant Figures

Number of significant figures indicates precision. Significant digits of a number are those that can be used with confidence, e.g., the number of certain digits plus one estimated digit.

53,800 How many significant figures?

5.38 x 104 35.380 x 104 45.3800 x 104 5

Zeros are sometimes used to locate the decimal point not significant figures.

0.00001753 40.0001753 40.001753 4

Error Definitions

Numerical error - use of approximations to represent exact mathematical operations and quantities

true value = approximation + error• error, t=true value - approximation• subscript t represents the true error• shortcoming....gives no sense of magnitude• normalize by true value to get true relative

error

Example 1:Find the bounds for the propagation in adding two numbers. For example if one is calculating X +Y where

X = 1.5 ± 0.05Y = 3.4 ± 0.04

SolutionMaximum possible value of X = 1.55 and Y = 3.44

Maximum possible value of X + Y = 1.55 + 3.44 = 4.99

Minimum possible value of X = 1.45 and Y = 3.36.

Minimum possible value of X + Y = 1.45 + 3.36 = 4.81

Hence 4.81 ≤ X + Y ≤4.99.

Example 2:

The strain in an axial member of a square cross-section is given by

Given

Find the maximum possible error in the measured strain.

Eh

F2

N9.072 Fmm1.04 hGPa5.170 E

http://numericalmethods.eng.usf.edu8

Example 2:

)1070()104(

72923

610286.64 286.64

Solution

Error definitions cont.

100valuetrue

errortruet

True relative percent error

Example

Consider a problem where the true answer is 7.91712. If you report the value as 7.92, answer the following questions.

1. What is the true error?2. What is the relative error?

Example Determine the absolute and

relative errors when approximating p by p∗ when

Solution

This example shows that the same relative error, 0.3333×10−1, occurs for widely varying absolute errors.

the absolute error can be misleading and the relative error more meaningful, because the relative error takes into consideration the size of the value.

Error Definitions cont. Round off error – Symmetric rounding

originate from the fact that computers retain only a fixed number of significant figures: y = 0.73248261 to be 0.7325

Error = y-round(y) = 0.00001739 and relative error = error /y = -0.000024

Truncation errors – Chopping errors that result from using an approximation in place of an exact mathematical procedure: y = 0.73248261 to be 0.7324

Error = 0.0008261 and relative error = 0.00011

Example Determine the five-digit (a) chopping and (b)

rounding values of the irrational number π. Solution The number π has an infinite decimal

expansion of the form π = 3.14159265. . . . Written in normalized decimal form, we have π = 0.314159265 . . . × 10.(a) The floating-point form of π using five-digit

chopping is f l(π) = 0.31415 × 10 = 3.1415.(b) The sixth digit of the decimal expansion of π is a 9,

so the floating-point form of π using five-digit rounding is

f l(π) = (0.31415 + 0.00001) × 10 = 3.1416.

Use absolute value. Computations are repeated until stopping

criterion is satisfied.

If the following criterion is met

you can be sure that the result is correct to at least n significant figures.

sa Pre-specified % tolerance based on the knowledge of your solution

)%10 (0.5 n)-(2s

Numerical Stability Rounding errors may accumulate

and propagate unstably in a bad algorithm.

Can be proven that for Gaussian elimination the accumulated error is bounded

Example Suppose that x = 57 and y = 13. Use five-digit

chopping for calculating x + y, x − y, x × y, and x ÷ y.

Solution: Note that

Solution (Cont.)

19

Chopping Errors (Error Bounds Analysis)

e

n

ennn

aaazfl

aaaaaaz

21

12121

.0)(

0,.0

Suppose the mantissa can only support n digits.

ennn

enn

nenn

enn

n

aaaaa

aa

z

zflz

aaaazflz

).0(

)0...00.0()(

).0()0...00.0()(

2121

21

2121zeroes

Suppose ß = 10 (base 10), what are the values of ai such that the errors are the largest?

Thus the absolute and relative chopping errors are

20

Chopping Errors (Error Bounds Analysis)

nezflz )(

n

z

zflz 1)(

nene

nn

nnn

aazflz

aaa

21

321

.0)(

1.0Because

nenee

ne

e

ne

enn

n

ne

ennn

ne

ennn

enn

aa

aaaaa

aaaaa

aa

z

zflz

1)1(1

21digits

2121

2121

21

1.0

100000.0

.0

.0

0...00.0)(

21

Round-off Errors (Error Bounds Analysis)

exponentbase)sign(1

0,.0 1121

e

aaaaaz enn

1n21

1n21

a2

])01...00.0().0[(

2a0).0(

)(e

n

en

n

aaa

aaa

zfl

Round down

Round up

fl(z) is the rounded value of z

22

Round-off Errors (Error Bounds Analysis) Absolute error of fl(z)

nennn

nennn

ennn

aaazflz

aaa

aaazflz

321

321

321

.0)(

.0

000.0)(

When rounding down

2

1)(.

2 11

nn aa nezflz 2

1)(

Similarly, when rounding up

12 na nezflz 2

1)(i.e.,

when

23

Round-off Errors (Error Bounds Analysis) Relative error of fl(z)

1

1

2121

2

1

)1.0()(.because)1(.2

1

)(.because)(.2

1

2

1)(2

1)(

n

n

en

e

n

ne

a

aazaa

zz

zflz

zflz

n

z

zflz 1

2

1)(

24

Summary of Error Bounds Analysis

β base n # of significant digits or # of digits in the mantissa

Regardless of chopping or round-off is used to round the numbers, the absolute errors may increase as the numbers grow in magnitude but the relative errors are bounded by the same magnitude.

Chopping Errors Round-off errors

Absolute

Relative n

z

zflz 1

2

1)(n

z

zflz 1)(

nezflz )( nezflz 2

1)(

25

Machine Epsilon

off-Round1

2

1)(eps

z

zflz n

Relative chopping error Chopping

1)(eps

z

zflz n

Relative round-off error

eps is known as the machine epsilon – the smallest number such that

1 + eps > 1

epsilon = 1;while (1 + epsilon > 1) epsilon = epsilon / 2;epsilon = epsilon * 2;

Algorithm to compute machine epsilon

26

Exercise

Discuss to what extent

(a + b)c = ac + bc

is violated in machine arithmetic.

27

How does a CPU compute the following functions for a specific x value?

cos(x) sin(x) ex log(x) etc.

Non-elementary functions such as trigonometric, exponential, and others are expressed in an approximate fashion using Taylor series when their values, derivatives, and integrals are computed.

Taylor series provides a means to predict the value of a function at one point in terms of the function value and its derivatives at another point.

28

Define the step size as h=(xi+1- xi), the series becomes:

)1()1(

)!1(

)(

n

n

n hn

fR

Taylor Series (nth order approximation):

The Reminder term, Rn, accounts for all terms from (n+1) to infinity.

nn

iii

n

iii

iii

ii Rxxn

xfxx

xfxx

xfxfxf )(

!

)()(

!2

)()(

!1

)()()( 1

)(2

1

"

1

'

1

nni

nii

ii Rhn

xfh

xfh

xfxfxf !

)(

!2

)(

!1

)()()(

)(2

"'

1

nth order approximation:

Second order approximation:

first order approximation

2"'

)(!2

)()(

!1

)()()( i

ii

ii xx

xfxx

xfxfxf

nn

ii

n

ii

ii

i Rxxn

xfxx

xfxx

xfxfxf )(

!

)()(

!2

)()(

!1

)()()(

)(2

"'

))(()()( 'iii xxxfxfxf

Any smooth function can be approximated as a polynomial.

Take x = xi+1 Then f(x) ≈ f(xi) zero order approximation

• Each additional term will contribute some improvement to the approximation. Only if an infinite number of terms are added will the series yield an exact result.

• In most cases, only a few terms will result in an approximation that is close enough to the true value for practical purposes

Taylor Series Expansion

1

11

1

......

321

!1

!

!3'''

!2''

'

iin

n

n

ii

nni

n

iiiii

xxhn

fR

xxsizestephwhere

Rhn

xf

hxf

hxf

hxfxfxf

Example

Use zero through fourth order Taylor series expansion to approximate f(1) given f(0) =

1.2 (i.e. h = 1) f x 01 015 0 5 0 25 1 24 3 2. . . . .x x x x

Note:f(1) = 0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 0.2 0.4 0.6 0.8 1

x

f(x)

Solution n=0

• f(1) = 1.2• = abs [(0.2 - 1.2)/0.2] x 100 = 500%

n=1• f '(x) = -0.4x3 - 0.45x2 -x -0.25• f '(0) = -0.25• f(1) = 1.2 - 0.25h = 0.95• t =375%

n=2• f "=-1.2 x2 - 0.9x -1• f "(0) = -1• f(1) = 0.45• t = 125%

n=3• f "'=-2.4x - 0.9• f "'(0)=-0.9• f(1) = 0.3• t =50%

Solution

n=4• f ""(0) = -2.4• f(1) = 0.2 EXACT

Why does the fourth term give us an exact solution?

The 5th derivative is zero In general, nth order polynomial, we get

an exact solution with an nth order Taylor series

Solution

35

Example

Approximate the function f(x) = 1.2 - 0.25x - 0.5x2 - 0.15x3 - 0.1x4

from xi = 0 with h = 1 and predict f(x) at xi+1 = 1.

Taylor Series ProblemUse zero- through fourth-order Taylor series

expansions to predict f(4) for f(x) = ln x using a base point at x = 2. Compute the percent

relative error t for each approximation.

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 1 2 3 4 5

x

y

37

Choose x = xi+1 and xi = 0 Then f(xi+1) = f(x) and (xi+1 – xi) = x

Since First Derivative of ex is also ex :

(2.) (ex )” = ex (3.) (ex)”’ = ex, … (nth.) (ex)(n) = ex

As a result we get:

Example: computing f(x) = ex using Taylor Series expansion

!...

!3!21

32

n

xxxxe

nx

Looks familiar?

nn

iii

n

iii

iii

ii Rxxn

xfxx

xfxx

xfxfxf )(

!

)()(

!2

)()(

!1

)()()( 1

)(2

1

"

1

'

1

Maclaurin series for ex

38

Choose x=xi+1 and xi=0 Then f(xi+1) = f(x) and (xi+1 – xi) = x

Derivatives of cos(x):

(1.) (cos(x) )’ = -sin(x) (2.) (cos(x) )” = -cos(x),

(3.) (cos(x) )”’ = sin(x) (4.) (cos(x) )”” = cos(x),

……

As a result we get:

Yet another example:computing f(x) = cos(x) using Taylor Series expansion

!6!4!2

1cos642 xxx

x

Error Propagation

Let xfl refer to the floating point representation of the real number x.

Since computer has fixed word length, there is a difference between x and xfl (round-off error)

and we would like to estimate the error in the calculation of f(x) :

)()()( flfl xfxfxf • Both x and f(x) are unknown. • If xfl is close to x, then we can use first order Taylor expansion and compute:

))(()()( flflfl xxxfxfxf

xxfxf flfl *)()(

Result: If f’(xfl) and x are known, then we can estimate the error using this formula