ES 272Numerical Analysis

ES 272

Introduction to

Numerical Analysis

What is Numerical AnalysisNA is the study of algorithms that use numerical approximation (as opposed to general symbolic representation) for the problems of mathematical analysis (Wikipedia)

• Before the advent of modern computers numerical methods often depended on hand interpolation on printed tables.

Babylonian clay tablet to calculate 2 (1600 BC)

History• As old as human civilization

– A root-finding method for solving a simple equation in The Rhind Mathematical Papyrus (Egypt, ~1650 BC).

– Archimedes (212 BC)’s “method of exhaustion” for calculating lengths, areas, and volumes.

• Modern Applications– Logarithms (Napier in 1614) a simple way of doing arithmetic

operations (addition, mult., div., exponents.)

• Numerical alternatives to Calculus problems (Newton, Leibnitz)– Many other scientist contributed later on (Euler, Lagrange)– Approximation techniques, numerical integration, solution of

differential equations, etc.Þ Applications in many different fields

• We use numerical methods when there is no explicit solution. We develop approximate solutions.

• More computer power better numerical models• Growth in power and availability of digital computers in

the last 50 years led to an increasing use of sophisticated mathematical models in sciences and engineering. Scientific Computing; Computational Sciences.

• Computer arithmetic– Historically varied among different computer manufacturers

problems problems of porting a software between different computers

– Standardized by IEEE around 90’s standards for computer floating-point arithmetic.

Some Examples• Natural Sciences

– Predicting climate change– Simulating a carbon nanotube

• Social Sciences– Statistical modeling

• Engineering– Designing an airplane– Building modern structures– Computer Aided Design (CAD)

• Medicine– Simulating a protein– Simulating reaction of chemicals

• Business– Optimization (inventory control, scheduling, storage, etc...)

• Many thousands of other applications...

Major concerns• Replacement of the problem that cannot be solved directly

with a “nearby problem” which can be solved more easily.• Widespread use of the language of linear algebra.• Error, its size and its analytical form; improve the

convergence behavior of the numerical solution.• Stability (sensitivity of the solution to small changes in the

data or the parameters). Stability must be established for most real life problems (unless it is a chaotic system).

• Precision of the computer arithmetic, especially a concern for large size numerical algebra solutions.

• Efficiency of the algorithms. What is the time (cost) of solution of n equations in Ax=b?

Numerical Linear Algebra• Solution of System of Equations (Ax=b)

– Direct methods (e.g. Gaussian elimination)– Indirect (iterative) methods

• Errors in calculation of x– Rounding errors finite length of numbers in computer

arithmetic.

• Non-linear problems – Reducing it to a sequence of a linear problem– Apply linear solution techniques

• Some examples– Optimization (Optimum x values for min(f(x))– Rootfinding (Approximate the function near the root)

Approximation Theory• Approximation for evaluating f(x) using basic

arithmetic operations (addition, subtraction, multiplication, division) and comparison operations (true/false)– Using four basic operations, find polynomial p(x) p(x)=a0 + a1x + .. + anxn

– Rational functions (polyn/polyn)– All other functions (x, 2x) must be reduced to polynomials

or rational functions

• Only finite number of operations are possible.• All function evaluations on calculators and computers

are performed in this manner.

Interpolation

• Interpolation is also a method of approximation– Finding a polynomial that satisfies a set of points– One can also use rational functions, trigonometric

polynomials, and spline functions.

• Numerical Integration and Differentiation– Approximation of integrals and derivatives are based on

interpolation.– Replace the function f(x) by an interpolating (a simpler)

function p(x)– Integrate or differentiate p(x)

Solving Differential and Integral Equations• Approximation theory + Solution of linear/non-linear

systems (often very large) • Most mathematical models in natural sciences and

engineering are based on ODE and PDE.– Finite element method (Replace the unknown function by a

simpler function, e.g. a polynomial)– Finite difference methods (Solving the problem in discrete

points, and approximating derivatives and integrals in discrete points)

Example: CAD for a Jet

Example: Temperature on a Jet

Example: Pressure on a Jet

Example: Computational Physics

Example: Computational Physics