Module

for

The Improved Newton Method

Introduction

Newton's method is used to locate roots of the equation . The Newton-

Raphson iteration formula is:

(1)

Given a starting value , the sequence is computed using:

(2) for

provided that .

If the value is chosen close enough to the root p, then the sequence generated in

(2) will converge to the root p. Sometimes the speed at which converges is

fast (quadratic) and at other times it is slow (linear). To distinguish these two cases we

make the following definitions.

Definition 1 (Order of Convergence) Assume that converges to p, and

set . If two positive constants exist, and

then the sequence is said to converge to p with order of convergence R. The

number A is called the asymptotic error constant. The cases are given

special consideration.

(3) If , and

then the convergence of is called quadratic.

(4) If , and

then the convergence of is called linear.

The mathematical characteristic for determining which case occurs is the "multiplicity"

of the root p.

Definition 2 (Order of a Root) If can be factored as

(5) where m is a positive integer,

and is continuous at and , then we say that has a root of

order m at .

A root of order is often called a simple root, and if it is called a multiple

root. A root of order is sometimes called a double root, and so on.

Theorem 1 (Convergence Rate for Newton-Raphson Iteration) Assume that

Newton-Raphson iteration (2) produces a sequence that converges to the

root p of the function .

(6) If p is a simple root, then convergence is quadratic and

for k sufficiently large.

(7) If p is a multiple root of order m, then convergence is linear and

for k sufficiently large.

Method A. Accelerated Newton-Raphson

Suppose that p is a root of order . Then the accelerated Newton-Raphson

formula is:

(8) .

Let the starting value be close to p, and compute the sequence

iteratively;

(9) for

Then the sequence generated by (9) will converge quadratically to p.

Proof Accelerated & Modified Newton-Raphson Accelerated & Modified

Newton-Raphson

On the other hand, if then one can show that the

function has a simple root at .

Derivation

Using in place of in formula (1) yields Method B.

Method B. Modified Newton-Raphson

Suppose that p is a root of order . Then the modified Newton-Raphson

formula is:

(10)

Derivation

Let the starting value be close to p, and compute the sequence

iteratively;

(11) for

Then the sequence generated by (11) converges quadratically to p.

Limitations of Methods A and B

Method A has the disadvantage that the order m of the root must be known a

priori. Determining m is often laborious because some type of mathematical analysis

must be used. It is usually found by looking at the values of the higher derivatives

of . That is, has a root of order m at if and only if

(12) .

Dodes (1978, pp. 81-82) has observed that in practical problems it is unlikely that we

will know the multiplicity. However, a constant m should be used in (8) to speed up

convergence, and it should be chosen small enough so that does not shoot to the

wrong side of p. Rice (1983, pp. 232-233) has suggested a way to empirically

find m. If is a good approximation to p and , and somewhat distant

from then m can be determined by the calculation:

.

Method B has a disadvantage, it involves three functions

. Again, the laborious task of finding the formula for could detract from

using Method B. Furthermore, Ralston and Rabinowitz (1978, pp. 353-356) have

observed that will have poles at points where the zeros of are not roots

of . Hence, may not be a continuous function.

The New Method C. Adaptive Newton-Raphson

The adaptive Newton-Raphson method incorporates a linear search method with

formula (8). Starting with , the following values are computed:

(13) for .

Our task is to determine the value m to use in formula (13), because it is not known a

priori. First, we take the derivative of in formula (5), and obtain:

(14)

When (5) and (14) are substituted into formula (1) we

have which can be simplified to get

.

This enables us to rewrite (13) as

(15) .

We shall assume that the starting value is close enough to p so that

(16) , where .

The iterates in (15) satisfy the following:

(17) for

If we subtract p from both sides of (17) then

and the result after simplification is:

(18) .

Since , the iterates get closer to p as j goes from , which is

manifest by the inequalities:

(19) .

The values are shown in Figure 1.

Notice that if the iteration (15) was continued for

then could be larger than . This is proven by

using the derivatives in (12) and the Taylor polynomial approximation of

degree m for expanded about :

(20) .

Figure 1. The values for the "linear search" obtained by using

formula (15)

near a "double root" p (of order ). Notice

that .

If is closer to p than then (19) and (20) imply that ,

hence we have:

(21) .

Therefore, the way to computationally determine m is to successively compute the

values using formula (13) for until we arrive

at .

The New Adaptive Newton-Raphson Algorithm

Start with , then we determine the next approximation as follows:

Observe that the above iteration involves a linear search in either the

interval when or in the interval when . In the

algorithm, the value is stored so that unnecessary computations are

avoided. After the point has been found, it should replace

and the process is repeated.

Example 1. Use the value and compare Methods A,B and C for finding the

double root of the equation .

Solution 1.

Behavior at a Triple Root

When the function has a triple root, then one more iteration for the linear search in

(13) is necessary. The situation is shown in Figure 2.

Figure 2. The values for the "linear search" obtained by using

formula (15)

near a "triple root" p (of order ). Notice that

.

Example 2. Use the value and compare Methods A,B and C for finding the

triple root of the equation .

Example 3. Use the value and compare Methods A,B and C for finding the

quadruple root of the equation

Acknowledgement

This module uses Mathematica instead of Pascal, and the content is that of the article:

John Mathews, An Improved Newton's Method, The AMATYC Review, Vol. 10, No. 2,

Spring, 1989, pp. 9-14.

References

1. Dodes, 1. A., Numerical analysis for computer science, 1978, New York: North-

Holland.

2. Mathews, J. H., Numerical methods for computer science, engineering and

mathematics, 1987, Englewood Cliffs, NJ: Prentice-Hall.

3. Ralston, A. & Rabinowitz, P., A first course in numerical analysis, second ed., 1978,

New York: McGraw-Hill.

4. Rice, J. R., Numerical methods, software and analysis: IMSL reference edition,

1983, New York: McGraw-Hill.

Module

for

The Bisection Method

Background. The bisection method is one of the bracketing methods for finding roots

of equations.

Implementation. Given a function f(x) and an interval which might contain a root,

perform a predetermined number of iterations using the bisection method.

Limitations. Investigate the result of applying the bisection method over an interval

where there is a discontinuity. Apply the bisection method for a function using an

interval where there are distinct roots. Apply the bisection method over a "large"

interval.

Theorem (Bisection Theorem). Assume that and that there exists a number

such that .

If have opposite signs, and represents the sequence of midpoints

generated by the bisection process, then

for ,

and the sequence converges to the zero .

That is, .

Proof Bisection Method Bisection Method

Mathematica Subroutine (Bisection Method).

Example 1. Find all the real solutions to the cubic equation .

Example 2. Use the cubic equation in Example 1 and perform the

following call to the bisection method.

Reduce the volume of printout.

After you have debugged you program and it is working properly, delete the

unnecessary print statements.

Concise Program for the Bisection Method

Now test the example to see if it still works. Use the last case in Example 1 given above

and compare with the previous results.

Example 3. Convergence Find the solution to the cubic equation

. Use the starting interval .

Solution 3.

Example 4. Not a root located Find the solution to the equation . Use the

starting interval .

Solution 4.

Animations (Bisection Method Bisection Method). Internet hyperlinks to

animations.

Old Lab Project (Bisection Method Bisection Method). Internet hyperlinks to an old

lab project.

Module

for

The Secant Method

The Newton-Raphson algorithm requires two functions evaluations per iteration,

and . Historically, the calculation of a derivative could involve

considerable effort. But, with modern computer algebra software packages such as

Mathematica, this has become less of an issue. Moreover, many functions have non-

elementary forms (integrals, sums, discrete solution to an I.V.P.), and it is desirable to

have a method for finding a root that does not depend on the computation of a

derivative. The secant method does not need a formula for the derivative and it can be

coded so that only one new function evaluation is required per iteration.

The formula for the secant method is the same one that was used in the regula falsi

method, except that the logical decisions regarding how to define each succeeding term

are different.

Theorem (Secant Method). Assume that and there exists a number

, where . If , then there exists a such that the

sequence defined by the iteration

for

will converge to for certain initial approximations .

Proof Secant Method Secant Method

Algorithm (Secant Method). Find a root of given two initial

approximations using the iteration

for .

Computer Programs Secant Method Secant Method

Mathematica Subroutine (Secant Method).

Example 1. Use the secant method to find the three roots of the cubic

polynomial .

Determine the secant iteration formula that is used.

Show details of the computations for the starting value .

Solution 1.

Example 2. Use Newton's method to find the roots of the cubic

polynomial .

2 (a) Fast Convergence. Investigate quadratic convergence at the simple root

, using the starting value

2 (b) Slow Convergence. Investigate linear convergence at the double root

, using the starting value

Solution 2.

The following subroutine call uses a maximum of 20 iterations, just to make sure

enough iterations are performed.

However, it will terminate when the difference between consecutive iterations is less

than .

By interrogating k afterward we can see how many iterations were actually performed.

Example 3. Fast Convergence Find the solution to .

Use the Secant Method and the starting approximations and .

Solution 3.

Example 4. Slow Convergence Find the solution to .

Use the Secant Method and the starting approximations and .

Solution 4.

Example 5. Convergence, Oscillation Find the solution to .

Use the Secant Method and the starting approximations and .

Solution 5.

Example 6. Convergence Find the solution to .

Use the Secant Method and the starting approximations and .

Solution 6.

Example 7. NON Convergence, Diverging to Infinity Find the solution to

.

Use the Secant Method and the starting approximations and .

Solution 7.

Example 8. Convergence Find the solution to .

Use the Secant Method and the starting approximations and .

Solution 8.

Animations (Secant Method Secant Method) . Internet hyperlinks to animations.

Research Experience for Undergraduates

Secant Method Secant Method Internet hyperlinks to web sites and a bibliography of

articles.

Download this Mathematica NotebooSecant MethodReturn to Numerical Methods

- Numerical Analysis

Module

for

The Regula Falsi Method

Background. The Regula Falsi method is one of the bracketing methods for finding

roots of equations.

Implementation. Given a function f(x) and an interval which might contain a root,

perform a predetermined number of iterations using the Regula Falsi method.

Limitations. Investigate the result of applying the Regula Falsi method over an interval

where there is a discontinuity. Apply the Regula Falsi method for a function using an

interval where there are distinct roots. Apply the Regula Falsi method over a "large"

interval.

Theorem (Regula Falsi Theorem). Assume that and that there exists a

number such that .

If have opposite signs, and

represents the sequence of points generated by the Regula Falsi process, then the

sequence converges to the zero .

That is, .

Proof False Position or Regula Falsi Method False Position or Regula Falsi

Method

Computer Programs False Position or Regula Falsi Method False Position or

Regula Falsi Method

Mathematica Subroutine (Regula Falsi Method).

Example 1. Find all the real solutions to the cubic equation .

Solution 1.

Remember. The Regula Falsi method can only be used to find a real root in an

interval [a,b] in which f[x] changes sign.

Example 2. Use the cubic equation in Example 1 and perform the

following call to the Regula Falsi subroutine.

Solution 2.

Reduce the volume of printout.

After you have debugged you program and it is working properly, delete the

unnecessary print statements.

Now test the example to see if it still works. Use the last case in Example 1 given above

and compare with the previous results.

Reducing the Computational Load for the Regula Falsi Method

The following program uses fewer computations in the Regula Falsi method and is the

traditional way to do it. Can you determine how many fewer functional evaluations are

used ?

Various Scenarios and Animations for Regula Falsi Method.

Example 3. Convergence Find the solution to the cubic equation

. Use the starting interval .

Solution 3.

Animations (Regula Falsi Method Regula Falsi Method). Internet hyperlinks to

animations.

Research Experience for Undergraduates

Regula Falsi Method Regula Falsi Method Internet hyperlinks to web sites and a

bibliography of articles.

Download this Mathematica Notebook Regula Falsi Method

Return to Numerical Methods - Numerical Analysis

Module

for

Gauss-Jordan Elimination

In this module we develop a algorithm for solving a general linear system of

equations consisting of n equations and n unknowns where it is assumed that the

system has a unique solution. The method is attributed Johann Carl Friedrich Gauss

(1777-1855) and Wilhelm Jordan (1842 to 1899). The following theorem states the

sufficient conditions for the existence and uniqueness of solutions of a linear system

.

Theorem (Unique Solutions) Assume that is an matrix. The following

statements are equivalent.

(i) Given any matrix , the linear system has a unique solution.

(ii) The matrix is nonsingular (i.e., exists).

(iii) The system of equations has the unique solution .

(iv) The determinant of is nonzero, i.e. .

It is convenient to store all the coefficients of the linear system in one array of

dimension . The coefficients of are stored in column of the array (i.e.

). Row contains all the coefficients necessary to represent the equation

in the linear system. The augmented matrix is denoted and the linear system

is represented as follows:

The system , with augmented matrix , can be solved by performing row

operations on . The variables are placeholders for the coefficients and cam be

omitted until the end of the computation.

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Theorem (Elementary Row Operations). The following operations applied to the

augmented matrix yield an equivalent linear system.

(i) Interchanges: The order of two rows can be interchanged.

(ii) Scaling: Multiplying a row by a nonzero constant.

(iii) Replacement: Row r can be replaced by the sum of that tow and a nonzero

multiple of any other row;

that is: .

It is common practice to implement (iii) by replacing a row with the difference of that

row and a multiple of another row.

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Definition (Pivot Element). The number in the coefficient matrix that is used to

eliminate where , is called the pivot element, and the

row is called the pivot row.

Theorem (Gaussian Elimination with Back Substitution). Assume that is an

nonsingular matrix. There exists a unique system that is equivalent to the given

system , where is an upper-triangular matrix with for

. After are constructed, back substitution can be used to solve for .

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Algorithm I. (Limited Gauss-Jordan Elimination). Construct the solution to the

linear system by using Gauss-Jordan elimination under the assumption that

row interchanges are not needed. The following subroutine uses row operations to

eliminate in column p for .

Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan

Elimination and Pivoting

Mathematica Subroutine (Limited Gauss-Jordan Elimination).

Example 1. Use the Gauss-Jordan elimination method to solve the linear

system .

Solution 1.

Example 2. Interchange rows 2 and 3 and try to use the Gauss-Jordan subroutine to

solve .

Use lists in the subscript to select rows 2 and 3 and perform the row interchange.

Solution 2.

Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan

Elimination and Pivoting

Mathematica Subroutine (Complete Gauss-Jordan Elimination).

Example 3. Use the improved Gauss-Jordan elimination subroutine with row

interchanges to solve .

Use the matrix A and vector B in Example 2.

Solution 3.

Example 4. Use the Gauss-Jordan elimination method to solve the linear

system .

Solution 4.

Use the subroutine "GaussJordan" to find the inverse of a matrix.

Theorem (Inverse Matrix) Assume that is an nonsingular matrix. Form the

augmented matrix of dimension . Use Gauss-Jordan elimination to

reduce the matrix so that the identity is in the first columns. Then the inverse

is located in columns .

Proof The Inverse Matrix The Inverse Matrix

Example 5. Use Gauss-Jordan elimination to find the inverse of the

matrix .

Solution 5.

Algorithm III. (Concise Gauss-Jordan Elimination). Construct the solution to the

linear system by using Gauss-Jordan elimination. The print statements are for

pedagogical purposes and are not needed.

Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan

Elimination and Pivoting

Mathematica Subroutine (Concise Gauss-Jordan Elimination).

Remark. The Gauss-Jordan elimination method is the "heuristic" scheme found in most

linear algebra textbooks. The line of code

divides each entry in the pivot row by its leading coefficient . Is this step

necessary? A more computationally efficient algorithm will be studied which uses

upper-triangularization followed by back substitution. The partial pivoting strategy will

also be employed, which reduces propagated error and instability.

Application to Polynomial Interpolation

Consider a polynomial of degree n=5 that passes through the six points

;

.

For each point is used to an equation , which in turn are used to

write a system of six equations in six unknowns

=

=

=

=

=

=

The above system can be written in matrix form MC = B

Solve this linear system for the coefficients and then construct the

interpolating polynomial

.

Example 6. Find the polynomial p[x] that passes through the six

points using the following steps.

(a). Write down the linear system MC = B to be solved.

(b). Solve the linear system for the coefficients using our Gauss-Jordan subroutine.

(c). Construct the polynomial p[x].

Solution 6.

Module

forJacobi and Gauss-Seidel Iteration

Background

Iterative schemes require time to achieve sufficient accuracy and are reserved for

large systems of equations where there are a majority of zero elements in the matrix.

Often times the algorithms are taylor-made to take advantage of the special structure

such as band matrices. Practical uses include applications in circuit analysis, boundary

value problems and partial differential equations.

Iteration is a popular technique finding roots of equations. Generalization of fixed

point iteration can be applied to systems of linear equations to produce accurate

results. The method Jacobi iteration is attributed to Carl Jacobi (1804-1851) and Gauss-

Seidel iteration is attributed to Johann Carl Friedrich Gauss (1777-1855) and Philipp

Ludwig von Seidel (1821-1896).

Consider that the n×n square matrix A is split into three parts, the main diagonal D,

below diagonal L and above diagonal U. We have A = D - L - U.

=

-

-

Definition (Diagonally Dominant). The matrix is strictly diagonally dominant if

for .

Theorem (Jacobi Iteration). The solution to the linear system can be

obtained starting with , and using iteration scheme

where

and .

If is carefully chosen a sequence is generated which converges to the

solution P, i.e. .

A sufficient condition for the method to be applicable is that A is strictly diagonally

dominant or diagonally dominant and irreducible.

Proof Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration

Theorem (Gauss-Seidel Iteration). The solution to the linear system can be

obtained starting with , and using iteration scheme

where

and .

If is carefully chosen a sequence is generated which converges to the

solution P, i.e. .

A sufficient condition for the method to be applicable is that A is strictly diagonally

dominant or diagonally dominant and irreducible.

Proof Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration

Computer Programs Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel

Iteration

Mathematica Subroutine (Jacobi Iteration).

Example 1. Use Jacobi iteration to solve the linear

system .

Try 10, 20 and 30 iterations.

Example 2. Use Jacobi iteration to attempt solving the linear

system .

Try 10 iterations.

Observe that something is not working. In example 5 we will check to see if this matrix

is diagonally dominant.

Example 3. Use Gauss-Seidel iteration to solve the linear

system .

Try 10, 20 iterations. Compare the speed of convergence with Jacobi iteration.

Example 4 Use Gauss-Seidel iteration to attempt solving the linear

system .

Try 10 iterations.

Observe that something is not working. In example 5 we will check to see if this matrix

is diagonally dominant.

.

Mathematica Subroutine (Gauss-Seidel Iteration).

Example 6. Use Jacobi and Gauss-Seidel iteration to solve the linear

system .

Use a tolerance of and a maximum of 50 iterations.

version of Jacobi iteration to solve the linear system .

Research Experience for Undergraduates

Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration Internet

hyperlinks to web sites and a bibliography of articles.

Download this Mathematica Notebook Jacobi and Gauss-Seidel Iteration

Return to Numerical Methods - Numerical Analysis

Module

for

Numerical Differentiation, Part I

Background.

Numerical differentiation formulas formulas can be derived by first constructing the

Lagrange interpolating polynomial through three points, differentiating the

Lagrange polynomial, and finally evaluating at the desired point. In this

module the truncation error will be investigated, but round off error from computer

arithmetic using computer numbers will be studied in another module.

Theorem (Three point rule for ). The central difference formula for the first

derivative, based on three points is

,

and the remainder term is

.

Together they make the equation , and the truncation

error bound is

where . This gives rise to the Big "O" notation for the error

term for :

.

Proof Numerical Differentiation Numerical Differentiation

Theorem (Three point rule for ). The central difference formula for the

second derivative, based on three points is

,

and the remainder term is

.

Together they make the equation , and the truncation

error bound is

where . This gives rise to the Big "O" notation for the error

term for :

.

Proof Numerical Differentiation Numerical Differentiation

Animations (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to animations.

Computer Programs Numerical Differentiation Numerical Differentiation

Project I.

Investigate the numerical differentiation formula and

truncation error bound

where . The truncation error is investigated. The round off

error from computer arithmetic using computer numbers will be studied in another

module.

Enter the three point formula for numerical differentiation.

Aside. From a mathematical standpoint, we expect that the limit of the difference

quotient is the derivative. Such is the case, check it out.

Example 1. Consider the function . Find the formula for the third

derivative , it will be used in our explorations for the remainder term and the

truncation error bound. Graph . Find the bound

. Look at it's graph and estimate the value , be sure to take the absolute value if

necessary.

Example 2 (a). Compute numerical approximations for the derivative , using

step sizes , include the details.

2 (b). Compute numerical approximations for the derivatives

, using step sizes .

2 (c). Plot the numerical approximation over the interval

. Compare it with the graph of over the interval .

Example 3. Plot the absolute error over the interval

, and estimate the maximum absolute error over the interval.

3 (a). Compute the error bound and observe

that over .

3 (b). Since the function f[x] and its derivative is well known, and we have the graph

for , we can observe that the maximum error on the given

interval occurs at x=0. Thus we can do better that "theory", we see that

over .

Example 4. Investigate the behavior of . If the step size is reduced by a factor

of then the error bound is reduced by . This is the behavior.

Project II.

Investigate the numerical differentiation

formulae and truncation error

bound where . The

truncation error is investigated. The round off error from computer arithmetic using

computer numbers will be studied in another module.

Enter the formula for numerical differentiation.

Aside. It looks like the formula is a second divided difference, i.e. the difference

quotient of two difference quotients. Such is the case.

Aside. From a mathematical standpoint, we expect that the limit of the second divided

difference is the second derivative. Such is the case.

Example 5. Consider the function . Find the formula for the fourth

derivative , it will be used in our explorations for the remainder term and the

truncation error bound. Graph . Find the bound

. Look at it's graph and estimate the value , be sure to take the absolute value if

necessary.

Example 6 (a). Compute numerical approximations for the derivatives

, using step sizes .

6 (b). Plot the numerical approximation over the interval

. Compare it with the graph of over the interval

Example 7. Plot the absolute error over the interval

, and estimate the maximum absolute error over the interval.

7 (a). Compute the error bound and observe

that over .

7 (b). Since the function f[x] and its derivative is well known, and we have the graph

for , we can observe that the maximum error on the given

interval occurs at x=0. Thus we can do better that "theory", we see that

over .

Example 8. Investigate the behavior of . If the step size is reduced by a factor

of then the error bound is reduced by . This is the behavior.

Example 9. Given , find numerical approximations to the

derivative , using two points and the forward difference formula.

Example 10. Given , find numerical approximations to the

derivative , using two points and the backward difference formula.

Example 11. Given , find numerical approximations to the

derivative , using two points and the central difference formula.

Example 12. Given , find numerical approximations to the

derivative , using three points and the forward difference formula.

Example 13. Given , find numerical approximations to the

derivative , using three points and the backward difference formula.

Example 14. Given , find numerical approximations to the

derivative , using three points and the central difference formula.

Example 15. Given , find numerical approximations to the second

derivative , using three points and the forward difference formula.

Example 16. Given , find numerical approximations to the second

derivative , using three points and the backward difference formula.

Example 17. Given , find numerical approximations to the second

derivative , using three points and the central difference formula.

Animations (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to animations.

Old Lab Project (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to an old lab project.

Research Experience for Undergraduates

Numerical Differentiation Numerical Differentiation Internet hyperlinks to web

sites and a bibliography of articles.

Module

for

The Trapezoidal Rule for Numerical Integration

Theorem (Trapezoidal Rule) Consider over , where .

The trapezoidal rule is

.

This is an numerical approximation to the integral of over and we have

the expression

.

The remainder term for the trapezoidal rule is , where

lies somewhere between , and have the equality

.

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical

Integration

Composite Trapezoidal Rule

An intuitive method of finding the area under a curve y = f(x) is by approximating

that area with a series of trapezoids that lie above the intervals . When

several trapezoids are used, we call it the composite trapezoidal rule.

Theorem (Composite Trapezoidal Rule) Consider over . Suppose

that the interval is subdivided into m subintervals of equal

width by using the equally spaced nodes

for . The composite trapezoidal rule for m subintervals is

.

This is an numerical approximation to the integral of over and we write

.

Remainder term for the Composite Trapezoidal Rule

Corollary (Trapezoidal Rule: Remainder term) Suppose that is subdivided

into m subintervals of width . The composite trapezoidal

rule

is an numerical approximation to the integral, and

.

Furthermore, if , then there exists a value c with a < c < b so that the

error term has the form

.

This is expressed using the "big " notation .

Remark. When the step size is reduced by a factor of the error term

should be reduced by approximately .

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical

Integration

Animations (Trapezoidal Rule Trapezoidal Rule).

Computer Programs Trapezoidal Rule for Numerical Integration Trapezoidal

Rule for Numerical Integration

Algorithm Composite Trapezoidal Rule. To approximate the integral

,

by sampling at the equally spaced points for

, where . Notice that and .

Mathematica Subroutine (Trapezoidal Rule).

Or you can use the traditional program.

Mathematica Subroutine (Trapezoidal Rule).

Example 1. Numerically approximate the integral by using the

trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

Example 2. Numerically approximate the integral by using the

trapezoidal rule with m = 50, 100, 200, 400 and 800 subintervals.

Example 3. Find the analytic value of the integral (i.e. find the

"true value").

Example 4. Use the "true value" in example 3 and find the error for the trapezoidal rule

approximations in example 2.

Example 5. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Example 6. Numerically approximate the integral by using

the trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

Example 7. Numerically approximate the integral by using

the trapezoidal rule with m = 50, 100, 200, 400 and 800 subintervals.

Example 8. Find the analytic value of the integral (i.e. find

the "true value").

Example 9. Use the "true value" in example 8 and find the error for the trapezoidal rule

approximations in exercise 7.

Example 10. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Recursive Integration Rules

Theorem (Successive Trapezoidal Rules) Suppose that and the

points subdivide into subintervals equal

width . The trapezoidal rules obey the relationship

.

Definition (Sequence of Trapezoidal Rules) Define

, which is the trapezoidal rule with step size . Then for each

define is the trapezoidal rule with step size

.

Corollary (Recursive Trapezoidal Rule) Start with . Then

a sequence of trapezoidal rules is generated by the recursive formula

for .

where .

The recursive trapezoidal rule is used for the Romberg integration algorithm.

Various Scenarios and Animations for the Trapezoidal Rule.

Example 11. Let over . Use the Trapezoidal Rule to

approximate the value of the integral.

Animations (Trapezoidal Rule Trapezoidal Rule).

Research Experience for Undergraduates

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical

Integration Internet hyperlinks to web sites and a bibliography of articles.

Trapezoidal Rule for Numerical Integration

Return to Numerical Methods - Numerical Analysis

Module

for

Simpson's Rule for Numerical Integration

The numerical integration technique known as "Simpson's Rule" is credited to the

mathematician Thomas Simpson (1710-1761) of Leicestershire, England. His also

worked in the areas of numerical interpolation and probability theory.

Theorem (Simpson's Rule) Consider over , where , and

. Simpson's rule is

.

This is an numerical approximation to the integral of over and we have

the expression

.

The remainder term for Simpson's rule is , where lies

somewhere between , and have the equality

.Simpson's Rule for

Numerical Integration Simpson's Rule for Numerical Integration

Composite Simpson Rule

Our next method of finding the area under a curve is by approximating that

curve with a series of parabolic segments that lie above the intervals

. When several parabolas are used, we call it the composite Simpson rule.

Theorem (Composite Simpson's Rule) Consider over . Suppose that

the interval is subdivided into subintervals of equal

width by using the equally spaced sample points

for . The composite Simpson's rule for subintervals is

.

This is an numerical approximation to the integral of over and we write

.

Simpson's Rule for Numerical Integration Simpson's Rule for Numerical

Integration

Remainder term for the Composite Simpson Rule

Corollary (Simpson's Rule: Remainder term) Suppose that is subdivided

into subintervals of width . The composite Simpson's

rule

.

is an numerical approximation to the integral, and

.

Furthermore, if , then there exists a value with so that the

error term has the form

.

This is expressed using the "big " notation .

Remark. When the step size is reduced by a factor of the remainder term

should be reduced by approximately .

Algorithm Composite Simpson Rule. To approximate the integral

,

by sampling at the equally spaced sample points

for , where . Notice that and .

Example 1. Numerically approximate the integral by using

Simpson's rule with m = 1, 2, 4, and 8.

Example 2. Numerically approximate the integral by using

Simpson's rule with m = 10, 20, 40, 80, and 160.

Example 3. Find the analytic value of the integral (i.e. find the

"true value").

Example 4. Use the "true value" in example 3 and find the error for the Simpson rule

approximations in example 2.

Example 5. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Example 6. Numerically approximate the integral by using

Simpson's rule with m = 1, 2, 4, and 8.

Example 7. Numerically approximate the integral by using

Simpson's rule with m = 10, 20, 40, 80, and 160 subintervals.

Example 8. Find the analytic value of the integral (i.e. find

the "true value").

Example 9. Use the "true value" in example 8 and find the error for the Simpson rule

approximations in example 7.

Example 10. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Various Scenarios and Animations for Simpson's Rule.

Example 11. Let over . Use Simpson's rule to

approximate the value of the integral.

Module

for

Riemann Sums, Midpoint Rule and Trapezoidal Rule

Definition. Definite Integral as a Limit of a Riemann Sum. Let be

continuous over the interval ,

and let be a partition, then the definite integral is given

by

,

where and the mesh size of the partition goes to zero in the "limit," i.e

. .

Riemann Sums Riemann Sums

Animations (Riemann Sums Riemann Sums).

Animations (Trapezoidal Rule Trapezoidal Rule).

Computer Programs Riemann Sums Riemann Sums

The following two Mathematica subroutines are used to illustrate this concept, which

was introduced in calculus.

Example 1. Let over . Use the left Riemann sum with n = 25, 50,

and 100 to approximate the value of the integral.

Example 2. Let over . Use the right Riemann sum with n = 25,

50, and 100 to approximate the value of the integral.

Example 3. Let over . Use the midpoint rule with n = 25, 50, and

100 to approximate the value of the integral.

Example 4. Let over . Use the trapezoidal rule with n = 25, 50,

and 100 to approximate the value of the integral.

.

Example 5. Let over . Compare the left Riemann sum, right

Riemann sum, midpoint rule and trapezoidal rule for n = 100 subintervals. Compare

them with the analytic solution.

Example 6. Let over .

6 (a) Find the formula for the left Riemann sum using n subintervals.

6 (b) Find the limit of the left Riemann sum in part (a).

Example 7. Let over .

7 (a) Find the formula for the right Riemann sum using n subintervals.

7 (b) Find the limit of the right Riemann sum in part (a).

Example 8. Let over .

8 (a) Find the formula for the midpoint rule sum using n subintervals.

8 (b) Find the limit of the midpoint rule sum in part (a).

Example 9. Let over .

9 (a) Find the formula for the trapezoidal rule sum using n subintervals.

9 (b) Find the limit of the trapezoidal rule sum in part (a).

Various Scenarios and Animations for Riemann Sums.

Example 10. Let over . Use the Left Riemann Sum to

approximate the value of the integral.

Example 11. Let over . Use the Right Riemann Sum

to approximate the value of the integral.

The following animations for the lower Riemann sums are included for illustration

purposes.

Example 12. Let over . Use the Lower Riemann Sum

to approximate the value of the integral.

The following animations for the upper Riemann sums are included for illustration

purposes.

Example 13. Let over . Use the Upper Riemann Sum

to approximate the value of the integral.

Example 14. Let over . Use the Midpoint Rule to

approximate the value of the integral.

Example 15. Let over . Use the Trapezoidal Rule to

approximate the value of the integral.

Module

for

Tri-Diagonal Linear Systems

Background

In applications, it is often the case that systems of equations arise where the

coefficient matrix has a special structure. Sparse matrices which contain a majority of

zeros occur are often encountered. It is usually more efficient to solve these systems

using a taylor-made algorithm which takes advantage of the special structure. Important

examples are band matrices, and the most common cases are the tridiagonal matrices.

Definition (Tridiagonal Matrix). An n×n matrix A is called a tridiagonal matrix

if whenever . A tridiagonal matrix has the for

(1) .

Theorem Suppose that the tri-diagonal linear system has

and for . If

, and , and , then

there exists a unique solution.

Proof Tri-Diagonal Matrices Tri-Diagonal Matrices

The enumeration scheme for the elements in (1) do not take advantage of the

overwhelming number of zeros. If the two subscripts were retained the computer would

reserve storage for elements. Since practical applications involve large values

of , it is important to devise a more efficient way to label only those elements

that will be used and conceptualize the existence of all those off diagonal zeros. The

idea that is often used is to call elements of the main diagonal , subdiagonal

elements are directly below the main diagonal, and the superdiagonal

elements are directly above the main diagonal. Taking advantage of the single

subscripts on we can reduce the storage requirement to merely

elements. This bizarre way of looking at a tridiagonal matrix is

.

Observe that A does not have the usual alphabetical connection with its new elements

named , and that the length of the subdiagonal and superdiagonal is n-1.

Definition (tri-diagonal system). If A is tridiagonal, then a tridiagonal system is

(2)

A tri-diagonal linear system can be written in the form

(3)

=

=

=

=

=

Goals

To understand the "Gaussian elimination process."

To understand solutions to"large matrices" that are contain mostly zero elements, called

"sparse matrices."

To be aware of the uses for advanced topics in numerical analysis:

(i) Used in the construction of cubic splines,

(ii) Used in the Finite difference solution of boundary value problems,

(iii) Used in the solution of parabolic P.D.E.'s.

Algorithm (tridiagonal linear system). Solve a tri-diagonal system given in

(2-3).

Below diagonal elements:

Main diagonal elements:

Above diagonal elements:

Column vector elements:

Computer Programs Tri-Diagonal Matrices Tri-Diagonal Matrices

Mathematica Subroutine (tri-diagonal linear system).

Example 1. Create a 31 by 31 tridiagonal matrix A where the main diagonal elements

are all "2" and all elements of the subdiagonal and superdiagonal elements are

"1." Then solve the linear system

where

. Compute the solution using decimal arithmetic and rational arithmetic.

Solution 1 (a).

Solution 1 (b).

Observation. The 31×31 matrix A in Example 1 has 961 elements, 870 of which are

zeros. This is an example of a "sparse" matrix. Do those zeros contribute to the

solution ? Since the matrix is tri-diagonal and diagonally dominant, there are other

algorithms which can be used to compute the solution. Our tri-diagonal subroutine will

work, and it requires only 12.3% as much space to save the necessary information to

solve the problem. In larger matrices the savings will be even more. It is more efficient

to use vectors instead of matrices when you have sparse matrices that are tri-diagonal.

Example 2. Solve the 31 by 31 system in Example 1 using our

TriDiagonal[a,d,c,b] subroutine. Compute the solution using decimal

arithmetic and rational arithmetic.

Example 3. Solve the 31 by 31 system in Example 1 using Mathematica's built

in TridiagonalSolve[a,d,c,b] procedure. Compute the solution using

decimal arithmetic and rational arithmetic.

Conclusion. If the matrix is tri-diagonal this structure should be considered because

the stored zeros do not contribute to the accuracy of the solution and just waste space in

the computer. Since tri-diagonal systems occur in the solution of boundary value

differential equations, practical problems could easily involve tri-diagonal matrices of

dimension 1000 x 1000. This would involve 1,000,000 entries if a square matrix was

used but only 2998 entries if the vector form of the tri-diagonal matrix is used.

Example 4. Solve the 31 by 31 system in Example 1 using the inverse matrix

and the computation .

Research Experience for Undergraduates

Tri-Diagonal Matrices Tri-Diagonal Matrices Internet hyperlinks to web sites and a

bibliography of articles.

Download this Mathematica Notebook Tri-Diagonal Linear Systems

Return to Numerical Methods - Numerical Analysis

(c) John H. Mathews 2004

(c) John H. Mathews 2004

(c) John H. Mathews 2004

(c) John H. Mathews 2004