Embed Size (px)
Citation preview
CHAPTER 3: ROOT FINDING– Bracketing Methods:– Bisection– False-Position
– Open Methods:– Newton-Raphson– Secant Method
– Roots of Polynomials:– Müller’s Method– Bairstow’s Method
– Packages and Libraries
“Roots” of this function represent the values of x that make f(x) equal to zero.
cbxaxxf 2)(
Consider the function:
0)( xf
Roots of f(x):
a
acbbx
2
42
2,1
There are many cases where roots can not be determined easily. In some cases, roots can not be determined analytically. e.g.,
5)( xexf x
We can use numerical methods to find the roots approximately.
Graphical technique:The most straightforward (and non-computer) technique is to
plot the function and see where it crosses the x-axis. Lack of precision
Guess a value of x and evaluate whether f(x) is zero. If not, make another guess... not efficient!
Trial-and –error:
Both approaches are very useful and supportive to give insight and confidence in numerical techniques of finding roots.
5)( xexf x
Two-curve graphical method:Another alternative is to devide the function into parts, e.g.
5)( xexf xxexf )(15)(2 xxf
)()( 21 xfxf
5)(2 xxf
xexf )(1
root
Consider the “falling object in air” problem:
t
m
gc
c
gmtv d
d
tanh)(
This equation cannot be solved for “m” explicitly. Then we call m as an implicit parameter.
In engineering many implicit parameter estimations are encountered.
To solve the equation for m:
vtm
gc
c
gmmf d
d
tanh)(
Find m values that makes f(m)=0 is the solution for m. A root finding problem!
Bracketing Methods
Open Methods
Bisection False-position
Secant Newton-Raphson
Müller’s method
Bairstow’s method
Roots of polynomials
(real roots of algebraic and non-algebraic eqn’s.)(real and complex roots of polynomials)
Algebraic equations, e.g., polynomial equations:
Non-algebraic equations, e.g., transcendental functions:
nnxaxaxaaxf ...)( 2
210
1ln)( 2 xxf
Major computational root finding methods
Bracketing Approach:A function typically changes its sign in the vicinity of a root.Two initial guesses are required. These guesses must “bracket”
the root (i.e., located on either side of the root).
0)()( ul xfxf
u
l
x
x Lower boundUpper bound
Increase x with a constant incremental length and calculate f(x). When the sign of the function changes at a particular interval,
divide the interval into smaller pieces. Choice of incremental length need to be optimized: Too small > cost of computation Too large > some roots may be missed
Incremental seach:
Graphical illustrations are useful to see the properties of the function and predicting the pitfalls of numerical approaches.
Behavior of functions around the root:
lx ux lx ux lx ux lx ux
lx ux
Function change sign if there are odd-number of roots.
Function does not change sign if there are even number of roots (or no roots)
Exception > multiple roots.multiple roots
Then f’(x) changes sign
Bisection Method In this method, when the sign of the function changes in the
incremental search between point xl and xu , i.e.
the interval is divided into half.
0)()( ul xfxf
2ul
r
xxx
The function is evaluated at xr.Location of the root is determined as
lying within the subinterval for the next iteration (xr is replaced either by xl or xu).
This process is repeated until the desired precision.
lx uxrx
Ex: Use bisection method to determine the mass of the falling object with a drag coefficient of 0.25 kg/m to have a velocity of 36 m/s after 4 s of free fall. (g=9.81 m/s2).
Initial guesses > xl=50 , xu=200
Lets replace > xl=xl xu=xr no sign changes
Then apply > xl=xr xu=xu
vtm
gc
c
gmmf d
d
tanh)( 0)( mf
1252
20050
rx
0)200()50( ff
%43.12%1007376.142
1257376.142
t
true value(not known!)
0)125()50( ff
5.1622
200125
rx %85.13t
xm
We can repeat the process to finer solutions:
Therefore the root is between 125 and 162.5 . Then the third iteration :
We can continue the process to the desired accuracy.
0)5.162()125( ff
75.1432
5.162125
rx %709.0t
Stopping criterion > we don’t know the true error!We can define an approximate error criterion based on the
previous and the current solution:
%100
newr
oldr
newr
a x
xx
Ex: Continue the previous solution until the approximate error falls below stopping criterion, e.g., s=0.5%.
n=1 xr=125
n=2 xr=162.5
%08.23%1005.162
1255.162
a
iteration xl xu xr |a |% |t |%
1 50 200 125 12.432 125 200 162.5 23.08 13.853 125 162.5 143.75 13.04 0.714 125 143.75 134.375 6.98 5.865 134.375 143.75 139.0625 3.37 2.586 139.0625 143.75 141.4063 1.66 0.937 141.4063 143.75 142.5781 0.82 0.118 142.5781 143.75 143.1641 0.41 0.30
Perc
ent r
elati
ve e
rror
|a|
|t|
Approximate error captures general trend of the true error.
It appears that approximate error is always greater than the true error.
Above is always true for the bisection method!
This allow us to impose the stopping criterion conveniently.
(i.e., root is known accurate to a prescribed value)
at
as
iterations
Rugged topography of the true error is due to the fact that the relative position of the root with respect to the lower and upper limits can lie anywhere within the bracketing interval for each iteration.
Approximate error gradually decreases, by definition, as the interval becomes smaller and smaller.
We know that the true root is between xl and xu:
lu xxx 22
lu xxx
Error:
Then, for the previous problem (after n=8 iterations), the root is accurate to:
5859.01641.143
2
5781.1427500.1431641.143
rx
Here we obtain an upper bound for the error. True error is smaller than this value.
A well-defined error analysis for the bisection method makes it more attractive compare to other root finding methods.
In bisection method, number of iterations (n) to reach the desired accuracy (Es) is known before the calculation:
nna
xE
2
0
sE
xn
0
2log
initial interval Test for the previous problem.
50lx200ux
%5.0)7376.142( sE
False Position MethodBasic assumption: Based on a
graphical insight, comparing f(xl) and f(xu), the root is expected to be closer to the smaller function value.
Intead of locating xr at the midpoint of the interval (bisection method), a straight line between two evaluation points of the function is drawn. The location of xr (false position) is where the line crosses the x-axis.
From similarity of the triangles:
lx ux
rx
)( lxf
)( uxf
)()(
))((
ul
uluur xfxf
xxxfxx
)(
)(
)(
)(
lr
ru
u
l
xx
xx
xf
xf
False position formula
Same approach with the bisection method is used, except, the next approximate root is calculated differently.
Same stopping criterion can be used:
)()(
))((
ul
uluur xfxf
xxxfxx
2
)( lur
xxx
Remember for bisection method:
%100
newr
oldr
newr
a x
xx
Ex: Use false position to solve the same problem of determining the mass of the falling object.
False-position method:
Initial guesses: xl=50 , xu=200
First iteration:
Second iteration:
Then;
50lx 579387.4)( lxf
200ux 860291.0)( uxf2773.176
860291.0579387.4
)20050(860291.0200
rx
592732.2)2773.176()50( ff
%5.23t
50lx 579387.4)( lxf
2773.176ux 566174.0)( uxf
3828.162566174.0579387.4
)2773.17650(566174.02773.176
rx%76.13a
%56.8t
Comparion of Bisection and False position:Pe
rcen
t rel
ative
err
or
False position
Bisection
iterations
True percent relative error for the problem of “finding the mass of the free falling object” using bisection and false-position methods are given below.
In false position method approximate root gradually converges to the true root (no raggedness).
Convergence of false position method is much faster than bisection method.
Cases Bisection is preferable to False position
1)( 10 xxf
Although false position generally performs faster than bisection, this is not always true. Consider, for example :
Here, we observe that convergence to the true root is very slow.
One of the bracketing limits tend to stay fixed which leads to poor convergence.
Note that, in the example, basic assumption of false-position that the root is closer to the smaller function evaluation value is violated.
x
)(xf
3.1ux
lx
Search the root for interval [0, 1.3]
rx 0.1
iteration xr a(%) t(%)
1 0.65 100 35
2 0.975 33.3 2.5
3 1.1375 14.3 13.8
4 1.05625 7.7 5.6
5 1.015625 4.0 1.6
iteration xr a(%) t(%)
1 0.09430 90.6
2 0.1876 48.1 81.8
3 0.26287 30.9 73.7
4 0.33811 22.3 66.2
5 0.40788 17.1 59.2
After five iterations bisection returns the root with less than 2% error. For false position the error is still 59% after same amount of iterations.
This shows that there can not be made a generalization for the root-finding methods.
A convenient check is to substitute the calculated root into the original equation whether the result is sufficiently close to zero. This check must be made for all root- finding algorithms.
Bisection False-position
!ta
Open methods:For bracketing methods the root is looked between in a interval-
described by a lower and upper bound. Bracket methods are convergent as they approaches to the root
as the iterations progress.Open methods are based on formulas. The starting point(s) do
not necessarily bracket the root.Open methods are prone to divergence- move away from the
root during the computation.However, when open methods converge, they do it much faster
than bracketing methods.
Simple fixed-point iteration (one-point iteration):Rearrange the equation f(x)=0 such that x is on the left side of the
equation, e.g.,0322 xx
can also be written as
2
32 x
x
The last function can be utilized such that the current value (xi+1) is calculated from the old value (xi), i.e.,
)(1 ii xgx 2
3)(
2 x
xgwhere
As with other iterative methods, the error can be defined as
%1001
1
i
iia x
xx
EX: Use simple fixed-point iteration to locate the root of starting an initial guess x0=0. (true value=0.56714329).
xexf x )(
Fixed point iteration shows characteristics of the linear convergence.
Newton Raphson MethodMost widely used method of root-finding algorithm.The process is as follows: 1. An initial guess xi is made. 2. Tangent line at xi is etrapolated
from point (xi, f(xi)) to the x-axis.
3. The point where the tangent crosses the x-axis is expected to represent an improved estimate of the root.
The slope of the tangent line is the derivative of the function at xi, i.e.,
))(,( ii xfx
root
)(
0)()(
1
'
ii
ii xx
xfxf
)(
)('1
i
iii xf
xfxx
Newton-Raphson formula
ix
)(' ixf
1ix
EX: Use Newton-Raphson method to estimate the root of
starting an inital guess of x0=0.
First derivative of the function:
Newton-Raphson formula:
As seen in the table, the method rapidly converges to the true root!
xexf x )(
1)(' xexf
11
i
i
xi
x
ii e
xexx
i xi t(%)
0 0 100
1 0.5000 11.8
2 0.566311003 0.147
3 0.567143165 0.0000220
4 0.567143290 < 10-8
Termination criteria:Error can be defined as
%1001
1
i
iia x
xx
That is, the error for the current iteration is in the order of the square of the error of the previous iteration.
It can be proved by Taylor theorem (see the book) that the relationship between the current and the previous error is:
)( 21 ii EOE
Error analysis for the last example:
Then
2,'
''
1, )(2
)(it
r
rit E
xf
xfE
(Quadratic convergence)
56714329.0rx
1)(' xexf xexf )(''
56714329.1)(' rxf 56714329.0)('' rxf
2,
2,1, 18.0
)56714329.1(2
56714329.0ititit EEE
0582.0)56714329.0(18095.0 21, tE
56714329.0056714329.00, tE 0008158.02, tE8
3, 1025.1 tE
154, 1083.2 tE
Pitfalls of the Newton-Raphson Method:
Inflection point in the vicinity of a root> divergence
Oscillations around a maximum/minimum> no convergence
0x1x
2x 0x 1x2x 3x
0x1x2x
Initial guess close to one root jumps to another root > solution jumps away
0x 1x
A zero slope is encountered> Solution shoots-off
When programming Newton-Raphson method:A plotting module should be included.At the end of the calculation, the result must be checked by
inserting it into the original function, whether the result is close to zero. (This is because although the program can return a very small a value, the solution may be far from the real root).
The program should include an upper limit on the number of iterations against possible oscillations, very slow convergences, or divergent solutions.
The program should check for possibility of f’(x)=0 during computation.
Secant MethodEvaluation of the derivative in Newton-Raphson method may not
always be straightforward. Derivative can be approximated by the (backward) finite
difference formula
ii
iii xx
xfxfxf
1
1' )()()(
))(,( ii xfx
root ix1ix
Secant method is very similar to Newton-Raphson method (an estimate of the root is predicted by extrapolating a tangent function to the x-axis) but the method uses a difference rather than a derivative.
1ix
Replacing the derivative in N-R formula by the finite difference formula yields:
)()(
))((
1
11
ii
iiiii xfxf
xxxfxx
Secant method formula
Note that two initial guesses are introduced (xi-1 and xi). However they do not have to bracket the root.
EX: Use the secant method to estimate the root of . Start with initial estimates of x-1=0 and x0=1.0.
second iteration:
xexf x )(
01 x
10 x0000.1)( 1 xf
63212.0)( 0 xf61270.0
)63212.0(1
)10)(632120.0(11
x %0.8t
11 x
61270.00 x63212.0)( 0 xf
07081.0)( 1 xf56384.0
)07081.0(63212.0
)61270.01)(07081.0(61270.02
x
%58.0t
Note similarity between F-P and Secant formulas:
Both uses two inital estimates and compute the slope of the function, and extrapolates to the x-axis.
third iteration:
61270.01 x56384.02 x
07081.0)( 1 xf
00518.0)( 2 xf
56717.0)00518.0(07081.0
)56384.061270.0)(00518.0(56384.03
x %0048.0t
Secant method versus False-position method
)()(
))((
1
11
ii
iiiii xfxf
xxxfxx
)()(
))((
ul
uluur xfxf
xxxfxx
Secant formula False-position formula
One critical difference is in the ways the previous estimate is replaced by the current estimate.
> false-position : sign-change to bracket the root > the method always converges.
> secant-method: follows a strict formula > no sign-change restriction > two values can be on the same side of the root > possible divergences.
rx
rx
)( uxf
)( lxf
)( uxf
)( lxf
False-positionn=1
False-positionn=2
rx
rx
)( ixf
)( 1ixf)( 1ixf
)( ixf
Secantn=1
Secantn=2
When it converges, Secant is superior to false-position method. This is due to the fact that in false-position method one end stays fixed to maintain for bracketing the root. This property is advenetegous in terms of preventing divergence. But it also results in a slower convergence.
True
Per
cent
rela
tive
erro
r
False position
Bisection
iterations
Secant
New
ton-Raphson
This figure compares the relative convergence rates of different root-finding algorithms.
Multiple roots:
(a double root at x=1)
)1)(1)(3()( xxxxf
1 3
)1)(1)(1)(3()( xxxxxf
Function touches the x-axis but does not cross it at the root.
(a triple root at x=1) 1 3
Function touches the x-axis and also crosses it at the root.
In general > Odd multiple roots cross the axis. > Even multiple roots do not cross the axis.
At even multiple roots the function does not change sign. This prevents use of bracketing methods > use open methods (be cautious of divergence)
At a multiple root, both f(x) and f’(x) goes to zero. > Problems with using Newton-Raphson and secant methods > Use the fact that f(x) reaches to zero before f’(x), so terminate computation before reaching f’(x)=0.
Newton-Raphson and Secant methods are linearly convergent to multiple roots > Need a small modification in the formula to make it quadratically convergenent (see the book):
Problems with Multiple roots:
)()()(
)()(''2'
'
1
iii
iiii
xfxfxf
xfxfxx
Modified Newton-Raphson equation for multiple root.
EX: Use standard and modified N-R methods to evaluate the multiple root of with an initial guess of x0=0.)1)(1)(3()( xxxxf
