Numerics of Special Functions, Nico M. Temme

Numerics of Special FunctionsNico M. Temme

Nico.Temme@cwi.nl

CWI, Amsterdam

Numerics of Special Functions, IMA, Special Functions in the Digital Age, July 2002 – p.1/65

Contents

Literature

Some simple experiences

Numerical aspects

Numerical methods

New approaches

Why is new software needed ?

What has to be done ?

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Contents

Literature

Numerical aspects

Numerical methods

New approaches

Present project

Literature

W. Gautschi (1975): Computational methods in specialfunctions - A survey.

Y.L. Luke (1969): The special functions and theirapproximations.

Y.L. Luke (1977): Algorithms for the computation ofspecial functions.

C.G. van der Laan & N.M. Temme (1984): Calculationof special functions.

D.W. Lozier & F.W.J Olver (1994) (and updates):Numerical evaluation of special functions.

Literature, Methods and Software

Baker (1992), Moshier (1989), Thompson (1997),Zhang & Jin (1996), Numerical Recipes

Maple, Mathematica, Matlab, Macsyma

Collections of algorithms: NAG, IMSL, SLATEC, CERN

Published algorithms: ACM, CPC, Applied Statistics

Repositories: GAMS at NIST, Netlib

How to compute an integral ?

Another integral

Exponential integral: Ei(x) or Ei(z) ?

Take a special case

Scaling

Extra elementary functions

Example: Beta integral

Example: Confluent hypergeometric function

Consider

F (λ) =

∫ ∞

−∞e−t2+2iλ

√t2+1 dt.

Maple 7, for λ = 10, gives

F (10) = −.1837516481 + .5305342893i.

With Digits = 40, the answer is

F (10) = −.1837516480532069664418890663053408790017+

+0.5305342892550606876095028928250448740020i.

Consider

F (λ) =

∫ ∞

−∞e−t2+2iλ

√t2+1 dt.

Maple 7, for λ = 10, gives

F (10) = −.1837516481 + .5305342893i.

With Digits = 40, the answer is

F (10) = −.1837516480532069664418890663053408790017+

+0.5305342892550606876095028928250448740020i.

Take another integral, which is almost the same:

F (λ) =

∫ ∞

−∞e−t2+2iλ

√t2+1 dt =⇒ G(λ) =

∫ ∞

−∞e−t2+2iλt dt.

Maple 7, for λ = 10, gives G(10) = −.1387778781 × 10−15.

With Digits = 40, the answer is G(10) = .16 × 10−42.

The correct answer is G(λ) =√

πe−λ2

and for λ = 10 wehave G(10) = 0.6593662989 × 10−43.

F (λ) =

∫ ∞

−∞e−t2+2iλ

√t2+1 dt =⇒ G(λ) =

∫ ∞

πe−λ2

and for λ = 10 wehave G(10) = 0.6593662989 × 10−43.

F (λ) =

∫ ∞

−∞e−t2+2iλ

√t2+1 dt =⇒ G(λ) =

∫ ∞

πe−λ2

and for λ = 10 wehave G(10) = 0.6593662989 × 10−43.

The message is: one should have some feeling about thecomputed result.

Otherwise a completely incorrect answer can be accepted.

Mathematica is more reliable here, and says:

"NIntegrate failed to converge to prescribed accuracy after7 recursive bisections in t near t = 2.9384615384615387".

By the way, ask Maple 7 to do the following integral

H(λ) =

∫ ∞

−∞e−t2+2iλ

√t2 dt,

and the funny answer is, after some simplification,

H(λ) =√

πe−λ2

[1 + signum(t) erf iλ],

where erf z is the error function.

Another integral

Consider

F (u) =

∫ ∞

0euit dt

t − 1 − i, u > 0.

Numerical quadrature gives F (2) = −0.934349 − 0.70922i.Mathematica 4.1 gives for u = 2 in terms of the MeijerG-function:

F (2) = πG2,12,3

0, 0, 12

; 2 − 2i

Mathematica evaluates: F (2) = −0.547745 − 0.532287i.

Ask Mathematica to evaluate F (u):

F (u) = eiu−uΓ(0, iu − u).

This gives F (2) = −0.16114 − 0.355355i.

So, we have three numerical results:

F1 = −0.934349 − 0.70922i,

F2 = −0.547745 − 0.532287i,

F3 = −0.16114 − 0.355355i.

Observe that F2 = (F1 + F3)/2. F1 is correct.

Maple:

F (u) = eiu−uEi(1, iu − u) = eiu−uΓ(0, iu − u)],

same as Mathematica. This is a wrong answer.

Next, Maple, after simplification, in terms of exponentialintegrals:

F (2) = e2i−2Ei(1, 2i − 2) + 2πie2i−2,

giving F (2) = −.9343493870 − .7092195099i, which is thecorrect answer.

Exponential integral: Ei(x) or Ei(z) ?

The exponential integrals are defined by

E1(z) =∞

tdt, |ph z| < π, Ei(x) = −−

tdt = −

−∞

tdt, x > 0,

Ei(x) = −∞

tdt = −E1(−x), x < 0.

Maple requires real x in Ei(x), as is in agreement with this definition.

Mathematica accepts complex z in Ei(z), although The Mathematica Book (4th Ed., p. 765)defines Ei(z) only for z > 0 by using a principal value integral. This is confusing.

The same happens in Gradshteyn & Ryzhik (Sixth Ed.): Ei(x) is only defined as above;

there is no proper definition of the exponential integral for complex argument.

Take a special case

Parabolic cylinder functions are special cases of the 1F1

functions, Kummer functions or Whittaker functions.The Mathematica Book (4th Ed., p. 765) advises to use

U(a, z) = 2−a/2z−1/2W−a/2,−1/4

but this is useless when z < 0.

Maple 7 uses the representations in terms of 1F1 functions,but this becomes very unstable when the parameters are"large".

Scaling

The function Γ∗(z) defined by

Γ(z) =√

2πzz−1/2e−zΓ∗(z),

Γ∗(z) ∼ 1 +1

288z2+ . . . , z → ∞,

can be computed within machine precision for almost allcomplex z. The precision in the gamma function itselffollows from the evaluation of the elementary function

√2πzz−1/2e−z.

To avoid underflow and overflow, and to control accuracy, itis very important to have scaled functions like Γ∗(z)available.

The same holds for Bessel functions, parabolic cylinderfunctions, and so on.The scaled Airy function Ai(z) defined by

Ai(z) = e−2

can be computed very accurate for complex z (not close tozeros of Ai(z)).

The scaling factor e−2

completely determines theaccuracy if z is large and complex.Again, scaled functions are very useful to avoid underflowand overflow, and to control accuracy.

Extra elementary functions

We have standard codes for

sin x, ln x, arctan x, ex, . . .

but usually not for

sin x − x

ln(1 + x)

arctan x − x

ex − 1

x, . . .

for small values of x. It is not difficult to write efficient codes(by using power series, for example). A standard packagefor this type of elementary functions would be very useful.

Example: Beta integral

The Beta integral can be written in the form

Γ(p)Γ(q)

Γ(p + q)=

√2π

√p + q

Γ∗(p)Γ∗(q)

Γ∗(p + q)ep ln(1− q

p+q)+q ln(1− p

When p + q is large, all quantities at the right-hand side canbe computed in good relative precision if codes for ln(1 + x)(for small x) and Γ∗(x) (for large x) are available.

Example: Confluent hypergeometric function

The standard solution of Kummer’s equation that is singularat the origin can be written in the form

U(a, c, z) =π

sin πc

[1F1(a; c; z)

Γ(1 + a − c)Γ(c)− z1−c 1F1(1 + a − c; 2 − c; z)

Γ(a)Γ(2 − c)

For small z this can be used for computations.

However, for integer values of c, problems arise.

A careful analysis is needed to avoid numericalcancellations.

We have

U(a, c, z) =∞∑

f0 =π

sinπc

Γ(1 + a − c)Γ(c)=

Γ(1 − c)

Γ(1 + a − c),

f1 =π

sin πc

Γ(1 + c)Γ(1 + a − c)− z−c

Γ(a)Γ(2 − c)

For small values of c the coefficient f1 is difficult to compute.Numerics of Special Functions, IMA, Special Functions in the Digital Age, July 2002 – p.20/65

An expansion in powers of c is not doable, because of allhigher derivatives of the gamma function.

The computation of f1 can be done if we have an algorithmfor

Γ(α − β)− 1

Γ(α + β)

[Γ(α + β)

Γ(α − β)− 1

for small values of |β|, α ∈ C.

The remaining fk can be obtained from recursions, if f1 isavailable.

Numerical aspects

Fixed precision or variable precision

Insight in function behavior

Selecting analytical tools

Selecting numerical methods

Stability of the algorithms

Efficiency of the algorithms

Underflow, overflow, scaling

Testing

High quality software

Fixed precision or variable precision

Fixed precision: efficiency

Variable precision: greater challenge

Insight in function behavior

Numerically satisfactory pair of solutions

Singular points, turning points

Influence of additional parameters

Stable representations

Selecting analytical tools

Are power series available ?

Are asymptotic expansions available ?

Are these uniform with respect to parameters ?

Are new expansions needed ?

Are integrals well conditioned ?

Are connection formulas available ?

Selecting numerical methods

Power series: convergent, asymptotic

Recursions, linear difference equations

Chebyshev expansions

Continued fractions

Quadrature: Gauss, trapezoidal

Uniform asymptotic expansions

Differential equations

Rational approximations, Padé, Chebyshev sense

Convergence acceleration

Stability of the algorithms

Rigorous error analysis ?

Connection formulas

An important source of errors: elementary functionswith large complex arguments

Efficiency of the algorithms

One universal algorithm ?

Power series if possible ?

Underflow, overflow, scaling

Avoid underflow or overflow by scaling

Discontinuous scaling factors may occur

Testing scaled functions: no guarantee

Testing

Wronskian relations

Contiguous relations

Other functional identities

Testing by using overlapping domains

Testing with multiple-precision algorithms

Comparison against a standard

Lozier (1996) Test service and reference software

High quality software

Refereed articles

Refereed software

Concern: maintenance

Numerical methods

Continued fractions

Quadrature, Gauss, trapezoidal

Rational approximations, Padé, Chebyshev sense

Estimating remainders

"Convergence" of asymptotic expansions

Domains: Where to use the series ?

Meaning of asymptotic expansions

Consider the expansion of the Kummer function:

1F1(a; c; z)

Γ(c)=

ezza−c

[R−1∑

(c − a)n(1 − a)nn! zn

+ O(|z|−R

Γ(c − a)

[S−1∑

(−1)n(a)n(1 + a − c)nn! zn

+ O(|z|−S

)]e±iπa,

This expansion is valid for large complex z in certainsectors, but also for positive z.

Observe that e±iπa is a complex quantity.If z > 0, a and c are real: Does this formula give a complexapproximation of a real function ?

1F1(a; c; z)

Γ(c)=

ezza−c

[R−1∑

(c − a)n(1 − a)nn! zn

+ O(|z|−R

Γ(c − a)

[S−1∑

(−1)n(a)n(1 + a − c)nn! zn

+ O(|z|−S

)]e±iπa

This expansion is valid for large complex z in certainsectors, but also for positive z.Observe that e±iπa is a complex quantity.

If z > 0, a and c are real: Does this formula give a complexapproximation of a real function ?

1F1(a; c; z)

Γ(c)=

ezza−c

[R−1∑

(c − a)n(1 − a)nn! zn

+ O(|z|−R

Γ(c − a)

[S−1∑

(−1)n(a)n(1 + a − c)nn! zn

+ O(|z|−S

)]e±iπa

This expansion is valid for large complex z in certainsectors, but also for positive z.Observe that e±iπa is a complex quantity.

If z > 0, a and c are real: Does this formula give a complexapproximation of a real function ?

First and second order difference equations

Stability analysis

Backward recursion

Nonlinear recursions: Gauss, Landen, AGM

Clenshaw, Luke: one variable, tabled coefficients

Luke: hypergeometric functions

(ωz)aU(a, c, ωz) =∞∑

Cn(z)T∗n(1/ω),

where T∗n is the shifted Chebyshev polynomial, 1 ≤ ω ≤ ∞,

z 6= 0, |ph z| < 3π/2.

The coefficients Cn(z) are known as Meijer’s G function,and Cn(z) satisfy a third order linear difference equation

If |ph z| < π the Cn(z) can be computed by using abackward recursion scheme.

Continued fractions

Upper and lower approximations

Transformations

Stopping criterion

Anomalous convergence (Gautschi (1977))

Quadrature, Gauss, trapezoidal

Gauss quadrature: for fixed precision

Trapezoidal rule: more flexible

Select suitable contours: avoid strong oscillations

G(λ) =

∫ ∞

−∞e−t2+2iλt dt = e−λ2

∫ ∞

−∞e−s2

Computation of coefficients

Domains: Where to use the expansions ?

Main approximants: higher transcendentals

For example, the Airy-type expansion for the J Besselfunction: as ν → ∞

Jν(νz) ∼ φ(ζ)

[Ai(ν2/3ζ)

∞∑

As(ζ)

Ai′(ν2/3ζ)

∞∑

Bs(ζ)

φ(ζ) =

1 − z2

3ζ3/2 = ln

1 +√

1 − z2

z−√

1 − z2.

Stability

Direction of integration

Parallel integration (Lozier & Olver (1993))

Higher order linear equations

Lanczos τ−method (Coleman, Rappoport)

Rational approximations, Pade, Chebyshev sense

Cody and co-workers: one variable rationalapproximations

Padé: applicable when coefficients can be obtainedeasily

Luke: tabled coefficients

Summing slowly convergent series

Summing divergent asymptotic series

Brezinski, Weniger

Special examples show impressive results

New approaches

Elementary functions: rigorous bounds

Multiple-precision computations

Unrestricted algorithms, error analysis

Methods for computing symmetric integrals

New packages for special functions

Elementary functions: rigorous bounds

Cody & Waite (1980)

Table-lookup algorithms ((Tang (1991), Rump (2001))

Zimmerman (France): The MPFR library(www.mpfr.org)

Cuyt & Verdonk (Belgium): continued fraction approach

Multiple-precision computations

Brent (1978): Fortran

Several packages in C, C++

Maple, Mathematica, ...

Methods for computing symmetric integrals

Main application: elliptic integrals.Carlson’s method: To iterate the duplication theorem andthen sum a five degree power series.

The symmetric integrals are of the form:

RF (x, y, z) =1

∫ ∞

0[(t + x)(t + y)(t + z)]−1/2 dt,

RJ (x, y, z) =3

∫ ∞

0[(t + x)(t + y)(t + z)]−1/2 (t + p)−1 dt,

The algorithms are very efficient and reliable.

New packages for special functions

Gautschi (1994): ORTHOPOL, also for Gauss-typequadrature rules

Koepf (1999): orthogonal polynomials, hypergeometricfunctions (numerics and symbolic)

Zeilberger (1990, ...) (and others): hypergeometricidentities

Are other software efforts still needed with the libraries ofMathematica and Maple available ?

A few remarks:

Refereed software

Repositories of free software

Fast algorithms in Fortran77, Fortran90, C, C++

Reliable software for large parameter cases

Appropriate scaling to avoid underflow and overflow

Complex variables

The land beyond Bessel

Integrals of special functions

q− special functions ?

Present project

(In collaboration with Amparo Gil and Javier Segura,Madrid).

Airy functions

Scorer functions

Kia(x), <Iia(x), x > 0, a ∈ IR

Parabolic cylinder functions for real arguments

Main tool: quadrature of integrals on complex contours,with saddle point analysis

Quadrature of integrals

A simple example is:

G(λ) =

∫ ∞

−∞e−t2+2iλt dt = e−λ2

∫ ∞

−∞e−s2

This makes sense, because

The new integral is real, without oscillations

The dominant term e−λ2

is in front of the new integral

The function Kia(x)

An integral representation:

Kia(x) =

∫ ∞

−∞e−x coshw+iaw dw.

Consider the case 0 ≤ a < x for which values Kia(x) > 0,although strong oscillations occur in the integral when a, xare both large.Deform the contour such that no oscillations occur.Write w = u + iv; we have

−x coshw + iaw = −x coshu cos v− av + i(−x sinh u sin v + au).

Take the imaginary part equal to zero.

v π/2v0

This happens when, see the figure,

v = arcsin(a

The contour runs through the saddle point at

v0 = arcsin(a

Integrate along this contour. Then, for 0 ≤ a < x,

Kia(x) =

∫ ∞

−∞e−φ(u)f(u) du = e−φ(0)

∫ ∞

−∞e−[φ(u)−φ(0)] du

φ(u) = x coshu cos v + av, φ(0) =√

x2 − a2 + a arcsin(a/x),

and f(u) = dwdu = 1 + i dv

The function φ(u) is positive. The factor e−φ(0) gives thedominant term in the asymptotic behaviour.

Trapezoidal rule

af(t) dt =

2h[f(a) + f(b)] + h

f (h j) + Rn, h =b − a

Compared with Gauss quadrature: very flexible;precomputed zeros and weights are not needed.

Error term, for some ξ ∈ (a, b):

Rn = −n h3

12f ′′(ξ).

Adaptive algorithm: use previous function values(h → h/2).

Trapezoidal rule

af(t) dt =

2h[f(a) + f(b)] + h

f (h j) + Rn, h =b − a

Rn = −n h3

12f ′′(ξ).

Trapezoidal rule

af(t) dt =

2h[f(a) + f(b)] + h

f (h j) + Rn, h =b − a

Rn = −n h3

12f ′′(ξ).

Example: fast convergence

Take as an example the Bessel function (h = π/n, x = 5)

π J0(x) =

∫ π

cos(x sin t) dt = h + h

n−1∑

cos [x sin(h j)] + Rn,

4 −.12 10−0

8 −.48 10−6

16 −.11 10−21

32 −.13 10−62

64 −.13 10−163

128 −.53 10−404

Much better than the estimate of Rn. Explanation:periodicity and smoothness.

The remainder: smooth and periodic

In fact we haveTheoremIf f(t) is periodic and has a continuous kth derivative, and ifthe integral is taken over a period, then

|Rn| ≤constant

Bessel function: we can take any k.See Luke (1969) (Vol. II, p. 218). Krumhaar (1965) derives:

|Rn| ≤ 2ex/2 (x/2)2n

(2n)!,

which is quite realistic for the value of x we chose.

The trapezoidal rule on IR

For integrals over IR the trapezoidal rule may again be veryefficient and accurate.Consider

∫ ∞

−∞f(t) dt = h

∞∑

j=−∞f(hj + d) + Rd(h)

where h > 0 and 0 ≤ d < h.

We use this for functions analytic in the strip:

Ga = {z = x + iy | x ∈ IR, −a < y < a}.

A class of analytic functions

Let Ha denote the linear space of functions f : Ga → C,which are bounded in Ga and for which

limx→±∞

f(x + iy) = 0

(uniformly in |y| ≤ a) and

M±a(f) =

∫ ∞

−∞|f(x ± ia)| dx =

limb↑a

∫ ∞

−∞|f(x ± ib)| dx < ∞.

The error is exponentially small

TheoremLet f ∈ Ha for some a > 0, and f even. Then

|Rd(h)| ≤ e−πa/h

sinh(πa/h)Ma(f),

for any y with 0 < y < a.

ProofThe proof is based on residue calculus.

See [13] (Vol. II, p. 217).

Example: modified Bessel function

Consider the modified Bessel function

K0(x) =1

∫ ∞

−∞e−x cosh t dt.

We have, with d = 0,

exK0(x) =1

2h + h

∞∑

e−x(cosh(hj)−1) + R0(h).

For x = 5 and several values of h we obtain (j0 denotes thenumber of terms used in the series)

h j0 R0(h)

1 2 −.18 10−1

1/2 5 −.24 10−6

1/4 12 −.65 10−15

1/8 29 −.44 10−32

1/16 67 −.19 10−66

1/32 156 −.55 10−136

1/64 355 −.17 10−272

Fast convergent; easy to program

We see in this example that, halving the value of h givesa doubling of the number of significant digits.

Roughly speaking, a doubling of the number of termsneeded in the series.

When programming this method, observe that whenhalving h, previous function values can be used.

Details on error bounds for the remainder in thisexample follow from Luke (1969).

Concluding remarks

There is a need for refereed software in the openliterature; further activities are needed.

Maple and Mathematica are doing a great job, inparticular in connection with multiple-precisionnumerics for special functions.

Sometimes the approaches in M&M are too general;the user should be alert when using these packages.

For large variables and complex variables, quadraturemethods for contour integrals are useful tools; ideasfrom asymptotic analysis are very fruitful here.

Concluding remarks

Thanks to ...

Thanks toFrédéric Goualard,

who developedprosper

A new LATEXclass to produce high quality slides

See also SIAM News, December 2001 andhttp://prosper.sourceforge.net

Thanks to ...

Thanks toFrédéric Goualard,

who developedprosper

A new LATEXclass to produce high quality slides

See also SIAM News, December 2001 andhttp://prosper.sourceforge.net

Numerics of Special Functions, Nico M. Temme

Documents

BG8 Numerics I - University of Maryland, Baltimore Countygobbert/kassel/modules_english.pdf · BG8 Numerics I Module name Numerics I Learning outcomes Students … have basic knowledge

Nico Nico Douga - Arranged for Piano Duet (Bass)

1 Shoehorning Apache Onto Your Box System Sizing Tips Sander Temme

Computer Algebra Systems: Numerics

Apache Performance Tuning Part 1: Scaling Up Sander Temme

Res Numerics

Bible Numerics - What is Truth?truthdefined.com/PDFs/Numerics-Allis.pdfBible Numerics 3 Oswald T. Allis Bible Numerics FIGURES are both interesting and fascinating. The almost infi-

The Finale Numerics Font eBook

Nico Ingles

Numerics PDE RD

Engineering Economics Monopoly Numerics

Diving in Sai Kung Issue & Gap Analysis Andries Temme Susan Voelkl

Civilian Use of Night Vision Goggles - AsMA · Civilian Use of Night Vision Goggles. GUILLERMO SALAZAR, LEONARD TEMME, AND J. CHARLES ANTONIO . SALAZAR . G, TEMME . L, ANTONIO . Jc

McGinley, Temme, and Geber 1989

Nico Presentation

Numerics and Boundary Conditions (used in PALM)palm.muk.uni-hannover.de/trac/chrome/site/tutorial/... · 2015-09-21 · Numerics Boundary Conditions Numerics and Boundary Conditions

BSDEs, Numerics and Finance

Information Security Systems > Security Aspects of Open Source Software Sander Temme

Elasticity, theory, applications, and numerics

Swiss Numerics Day 2019